Wei Lin

I am a research associate at the Institute for Machine Learning headed by Prof. Sepp Hochreiter (Father of LSTM) at the Johannes Kepler University (JKU) Linz.

I did my PhD at the Institute of Computer Graphics and Vision, Graz University of Technology (TU Graz ICG) in Austria, supervised by Prof. Horst Bischof (Professor and the Rector of TU Graz) and Prof. Hilde Kuehne (Tuebingen AI Center, University of Tuebingen, MIT-IBM Watson AI Lab)

I also work in close collaboration with Leonid Karlinsky and Rogerio Feris (Principal scientists and research manager) from the MIT-IBM Watson AI Lab.

Previously, I received my Master's degree in Electrical and Computer Engineering at the Technical University of Munich in Germany.

I like traveling 🥾 and learning languages 🌍 I speak English (C1), German (B2), Chinese (mother tongue) and some French.

Email / CV / Google Scholar / Github / LinkedIn / Twitter

News

09/2024 Our paper ConMe is accepted at NeurIPS 2024 Datasets & Benchmarks Track!
08/2024 Our paper Towards Multimodal In-Context Learning for Vision & Language Models is accepted at ECCV 2024 Workshop on Multimodal Agents!
07/2024 Our paper ViLGOD is accepted as an Oral Presentation at BMVC 2024!
07/2024 Our paper Meta prompting is accepted to ECCV 2024!
01/2024 I am happy to announce that I will co-organize the 2nd Workshop on "What is Next in Multimodal Foundation Models?" as a program chair and challenge chair on CVPR 2024 in Seattle! Follow us on X (Twitter) and stay tuned!
10/2023 I joined the Institute for Machine Learning headed by Prof. Sepp Hochreiter (Father of LSTM) at the Johannes Kepler University (JKU) Linz!
09/2023 One paper is accepted to NeurIPS 2023!
09/2023 Our ICCV paper MAtch, eXpand and Improve (MAXI) is accepted as an Oral Presentation at the ICCV 2023 Workshop PerDream: PERception, Decision making and REAsoning through Multimodal foundational modeling!
07/2023 Two papers are accepted to ICCV 2023!
07/2023 I attended the International Computer Vision Summer School 2023!
04/2023 Application to the International Computer Vision Summer School 2023 is accepted (acceptance rate 27%)!
03/2023 Application to the CVPR 2023 Doctoral Consortium is accepted (acceptance rate 13%)!
03/2023 One paper is accepted to Robotics and Automation Letters 2023!
02/2023 Two papers are accepted to CVPR 2023!
07/2022 One paper is accepted to ECCV 2022!

Research

I am interested in computer vision and machine learning in general. My research is mainly about vision-language models, multimodal learning, domain adaptation and video understanding.

	Comparison Visual Instruction Tuning Wei Lin, Muhammad Jehanzeb Mirza, Sivan Doveh, Rogerio Feris, Raja Giryes, Sepp Hochreiter, Leonid Karlinsky In collaboration with the MIT-IBM Watson AI Lab Arxiv, 2024 arxiv / 🤗 Dataset / code / video an approach for collection of visual instructions that improves Commonality and difference spoting capabilities for Large Multimodal Modes
	Conme: Rethinking Evaluation of Compositional Reasoning for Modern VLMs Irene Huang, Wei Lin, Muhammad Jehanzeb Mirza, Jacob Hansen, Sivan Doveh, Victor Ion Butoi, Roei Herzig, Assaf Arbelle, Hilde Kuehne, Trevor Darrell, Chuang Gan, Aude Oliva, Rogerio Feris, Leonid Karlinsky (equal contribution) In collaboration with the MIT-IBM Watson AI Lab NeurIPS, 2024 Datasets & Benchmarks Track arxiv / 🤗 Dataset / code / video
	Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs Muhammad Jehanzeb Mirza, Leonid Karlinsky, Wei Lin, Sivan Doveh, Jakub Micorek, Mateusz Kozinski, Hilde Kuehne, Horst Possegger In collaboration with the MIT-IBM Watson AI Lab ECCV, 2024 arxiv / code / video
	Vision-Language Guidance for LiDAR-based Unsupervised 3D Object Detection Christian Fruhwirth-Reisinger, Wei Lin, Dusan Malic, Horst Bischof, Horst Possegger BMVC, 2024 Oral Presentation arxiv / code / video
	Towards Multimodal In-Context Learning for Vision & Language Models Sivan Doveh, Shaked Perek, Muhammad Jehanzeb Mirza, Wei Lin, Amit Alfassy, Assaf Arbelle, Shimon Ullman, Leonid Karlinsky ECCV 2024 Workshop on Multimodal Agents arxiv / code / video
	Overlooked Aspects in the Evaluation of Out-Of-Distribution Detection Methods Bernhard Lehner, Christian Huber, Bernhard Moser, Claus Hofmann, Wei Lin, Sepp Hochreiter (*equal contribution) Arxiv, 2024 arxiv / code / video
	LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collections Muhammad Jehanzeb Mirza, Leonid Karlinsky, Wei Lin, Mateusz Kozinski, Horst Possegger, Rogerio Feris, Horst Bischof NeurIPS, 2023 arxiv / code / video
	MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge Wei Lin, Leonid Karlinsky, Nina Shvetsova, Horst Possegger, Mateusz Kozinski, Rameswar Panda, Rogerio Feris, Hilde Kuehne, Horst Bischof In collaboration with the MIT-IBM Watson AI Lab ICCV, 2023 arxiv / code / video Unsupervised finetuning of Vision-Language models for zero-shot and few-shot action recognition, with GPT3 text expansion and video frame captioning.
	TAP: Targeted Prompting for Task Adaptive Generation of Textual Training Instances for Visual Classification Muhammad Jehanzeb Mirza, Leonid Karlinsky, Wei Lin, Horst Possegger, Rogerio Feris, Horst Bischof Arxiv, 2023 arxiv / code / video
	MATE: Masked Autoencoders are Online 3D Test-Time Learners Muhammad Jehanzeb Mirza, Inkyu Shin, *Wei Lin, Andreas Schriebl, Kunyang Sun, Jaesung Choe, Horst Possegger, Mateusz Kozinski, In So Kweon, Kun-Jin Yoon, Horst Bischof (equal contribution) ICCV*, 2023 arxiv / code / video
	Video Test-Time Adaptation for Action Recognition Wei Lin, Muhammad Jehanzeb Mirza, Mateusz Kozinski, Horst Possegger, Hilde Kuehne, Horst Bischof (equal contribution) CVPR*, 2023 arxiv / 🤗 Dataset / code / video Test-time adaptation of video action recognition against common distribution shifts.
	ActMAD: Activation Matching to Align Distributions for Test-Time-Training Muhammad Jehanzeb Mirza, Pol Jané Soneira, Wei Lin, Mateusz Kozinski, Horst Possegger, Horst Bischof CVPR, 2023 arxiv / code / video
	Unsupervised Class-aware 3D Object Detection in LiDAR Point Clouds Christian Fruhwirth-Reisinger, Wei Lin, Dusan Malic, David Schinagl, Georg Krispel, Horst Possegger, Horst Bischof Arxiv, 2023 arxiv / code / video
	CycDA: Unsupervised Cycle Domain Adaptation to Learn from Image to Video Wei Lin, Anna Kukleva, Kunyang Sun, Horst Possegger, Hilde Kuehne, Horst Bischof ECCV, 2022 paper / arxiv / code / video Unsupervised image-to-video domain adaptation.
	Extended Abstract CycDA: Unsupervised Cycle Domain Adaptation to Learn from Image to Video Wei Lin, Anna Kukleva, Kunyang Sun, Horst Possegger, Hilde Kuehne, Horst Bischof ECCV Workshop of Out Of Distribution Generalization in Computer Vision, 2022 paper / code / video
	AIR-DA: Adversarial Image Reconstruction for Unsupervised Domain Adaptive Object Detection Kunyang Sun, Wei Lin, Haoqin Shi, Zhengming Zhang, Yongming Huang, Horst Bischof IEEE Robotics and Automation Letters (RA-L) 2023 paper / arxiv / code / video
	TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clustering Wei Lin, Anna Kukleva, Horst Possegger, Hilde Kuehne, Horst Bischof Computer Vision Winter Workshop, 2023 arxiv / code / video
	Sit Back and Relax: Learning to Drive Incrementally in All Weather Conditions Stefan Leitner, Muhammad Jehanzeb Mirza, Wei Lin, Jakub Micorek, Marc Masana, Mateusz Kozinski, Horst Possegger, Horst Bischof Intelligent Vehicle Conference, 2023 arxiv / code / video

Academic Service

Conference Reviewer: ECCV 2022, ISMAR 2023, CVPR 2023, NeurIPS 2023, WACV 2024, CVPR 2024, ECCV 2024, NeurIPS 2024, NeurIPS 2024 Dataset and Benchmark Track
Journal Reviewer: TPAMI 2023, TNNLS 2023, IEEE Trans. Multimedia 2023, Pattern Recognition Letters 2024

Teaching

Deep Learning and Neural Networks I - Exercise
Machine Learning: Supervised Techniques - Exercise
Deep Learning and Neural Networks II - Exercise
Machine Learning: Unsupervised Techniques - Exercise

Activity

International Computer Vision Summer School 2023

template from Jon Barrion