TY - GEN
T1 - Captioning Bosch: A Twitter Bot
AU - Ferner, C.
N1 - Conference code: 182301
Export Date: 14 December 2023
Correspondence Address: Ferner, C.; Salzburg University of Applied SciencesAustria; email: [email protected]
Funding details: 20102-F1901166-KZP, WISS 2025
Funding text 1: This project was conducted at the Applied Data Science Lab of the Salzburg University of Applied Sciences under its doctoral support programme and is partially funded by the Science and Innovation Strategy Salzburg (WISS 2025) project ”IDA-Lab Salzburg”, grant number 20102-F1901166-KZP.
References: Benson, Louise, Pointless Profundity? How Twitter Art Bots Are Shaking Up Visual Culture (2019) Elephant, , https://elephant.art/pointless-profundity-twitter-artbots-shaking-visual-culture/, Last accessed on 2022-04-27; Chen, Xinlei, Fang, Hao, Lin, Tsung-Yi, Vedantam, Ramakrishna, Gupta, Saurabh, Dollár, Piotr, Lawrence Zitnick, C, (2015) Microsoft COCO Captions: Data Collection and Evaluation Server, , arXiv preprint arXiv:1504.00325; Chu, Zi, Gianvecchio, Steven, Wang, Haining, Jajodia, Sushil, Detecting Automation of Twitter Accounts: Are You a Human, Bot, or Cyborg? (2012) IEEE Transactions on Dependable and Secure Computing, 9 (6), pp. 811-824; Cornia, Marcella, Stefanini, Matteo, Baraldi, Lorenzo, Cucchiara, Rita, Meshed-memory Transformer for Image Captioning (2020) Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10578-10587; Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L., ImageNet: A Large-Scale Hierarchical Image Database (2009) CVPR09. IEEE; Dosovitskiy, Alexey, Beyer, Lucas, Kolesnikov, Alexander, Weissenborn, Dirk, Zhai, Xiaohua, Unterthiner, Thomas, Dehghani, Mostafa, Houlsby, Neil, An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2021) International Conference on Learning Representations. ICLR; Kelly Fitzpatrick, L., (2017) Anatomy of a Museum Twitter Bot. Medium, , https://medium.com/berkman-klein-center/anatomyof-a-museum-twitter-bot-2311d81de243, Last accessed on 2022-04-27; McMullan, Thomas, This Bonkers Bot is the Only Twitter Account Worth Following Right Now (2018) Wired, , https://www.wired.co.uk/article/bosch-twitter-bot-digital-art, Last accessed on 2022-04-27; Radford, Alec, Wu, Jeffrey, Child, Rewon, Luan, David, Amodei, Dario, Sutskever, Ilya, Language Models Are Unsupervised Multitask Learners (2019) OpenAI Blog, 1 (8), p. 9; Varol, Onur, Ferrara, Emilio, Davis, Clayton, Menczer, Filippo, Flammini, Alessandro, Online Human-Bot Interactions: Detection, Estimation, and Characterization (2017) Eleventh International AAAI Conference on Web and Social Media, pp. 280-289. , AAAI; Xu, Kelvin, Ba, Jimmy, Kiros, Ryan, Cho, Kyunghyun, Courville, Aaron, Salakhudinov, Ruslan, Zemel, Rich, Bengio, Yoshua, Show, Attend and Tell: Neural Image Caption Generation with Visual Attention (2015) Proceedings of the 32nd International Conference on Machine Learning, pp. 2048-2057. , PMLR
PY - 2022
Y1 - 2022
N2 - The artworks by Dutch painter Hieronymus Bosch are well known for their incredible wealth of details. The popular BoschBot regularly posts small segments of the digitized paintings on Twitter, thus relieving their density and making them more accessible. CaptioningBoschBot, the Twitter bot presented in this demo, reverses the creative process of the artist: It uses the out-of-context painting segments as input for an encoder-decoder model to generate captions that interpret the painted objects. As the model was only trained on realistic, photographic images, curious interpretations of the otherworldly details can be observed. The generated captions are again posted on Twitter to encourage discussions about Bosch's masterpieces and the AI technology in general. © 2022 International Joint Conferences on Artificial Intelligence. All rights reserved.
AB - The artworks by Dutch painter Hieronymus Bosch are well known for their incredible wealth of details. The popular BoschBot regularly posts small segments of the digitized paintings on Twitter, thus relieving their density and making them more accessible. CaptioningBoschBot, the Twitter bot presented in this demo, reverses the creative process of the artist: It uses the out-of-context painting segments as input for an encoder-decoder model to generate captions that interpret the painted objects. As the model was only trained on realistic, photographic images, curious interpretations of the otherworldly details can be observed. The generated captions are again posted on Twitter to encourage discussions about Bosch's masterpieces and the AI technology in general. © 2022 International Joint Conferences on Artificial Intelligence. All rights reserved.
KW - Artificial intelligence
KW - Social networking (online)
KW - AI Technologies
KW - Creative process
KW - Encoder-decoder
KW - Photographic image
KW - Photography
U2 - 10.24963/ijcai.2022/694
DO - 10.24963/ijcai.2022/694
M3 - Conference contribution
SP - 5011
EP - 5014
BT - Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22)
PB - International Joint Conferences on Artificial Intelligence
T2 - 31st International Joint Conference on Artificial Intelligence, IJCAI 2022
Y2 - 23 July 2022 through 29 July 2022
ER -