Yann LeCun Paper Rejected – Power Of Double Blind Review
Yann Andre LeCun, a French computer scientist who focuses on machine learning, computer vision, mobile robotics and computational neuroscience, recently tweeted that one of his articles was rejected from NeurIPS 2021.
Yann LeCun is a professor of silver at the Courant Institute of Mathematical Sciences at New York University and vice president, chief AI scientist at Facebook. He is well known for his work on optical character recognition and computer vision using convolutional neural networks (CNNs) and is often considered the inventor of convolutional networks. He is also the co-creator of the DjVu image compression technology. In addition, LeCun and Leon Bottou co-created the Lush programming language. The author is a multi-faceted individual with academic and industrial background in artificial intelligence, machine learning, deep learning, computer vision, intelligent data analysis, data mining, data compression, digital library systems, and robotics.
Yann recently co-wrote an article with Adrien bardes and Jean Ponce in NeurIPS entitled “VICReg: Regularization of variance-invariance-covariance for self-supervised learning” (ArXiv link), which has received 12 citations since May. However, to everyone’s surprise, the article was rejected.
VICReg is a simple self-supervised image representation learning approach that breaks down the problem into three distinct principles: learning invariance at different views with an invariance term, avoiding collapse of representations with a regularization term of variance and disseminate information through the different dimensions of the representations with a covariance regularization term.
On several downstream tasks, VICReg achieves state-of-the-art results, pushing the boundaries of non-contrast self-supervised learning.
Image source: VICReg.
The computation of the covariance matrix for each processed batch, which is quadratic in the dimension of the projection vectors, determines the computational and memory cost of VICReg. Experiments have shown that increasing the size of the projections greatly increases performance, indicating the need for other redundancy reduction methods that do not rely on the full computation of the covariance matrix.
Image source: Table 2 Results
When used with methods such as SimSiam, the term hinge-over-variance prevents collapse and eliminates the requirement for a batch standard or predictor.
Image source: Table 3
The authors said future research would examine how different approximation techniques and entirely new redundancy reduction methodologies based on higher-order statistics could be used to overcome this quadratic barrier.
How legends cope with adversity
In response to one of the questions on Linkedin, “Is it possible for neurips2021 reviewers to reject an article written by Prof. Yann LeCun? since he is widely regarded as one of the best minds in the field of artificial intelligence ”, he replied in the following way:“ Over the years, a number of articles have been rejected by NeurIPS. NeurIPS reviews are double-blind, which means reviewers do not know the identity of the authors. This is generally a good thing: an article should not be accepted solely on the basis of the fame of one of the authors. Indeed, many of my rejected articles were correctly rejected: they were written by students and did not impress me. However, it was beneficial for the students to receive feedback from the assessors. A handful of articles that were rejected were ones that I found extremely good and fascinating. One of them is this one.
Academic publishing is known for its rejection. Many researchers were waiting to see if their papers would be accepted for this year’s NeurIPS conference. Articles are rejected for a variety of reasons, ranging from easily avoidable errors and omissions to simply beyond the scope of the journal. Even if a manuscript appears to be “of no interest to anyone,” it may be of interest to someone. No one wants to see their article unpublished after years of research and months of writing and formatting a well-produced research article.
LeCun wears many feathers in his hat. A member of the National Academies of Sciences and Engineering of the United States of America and of the National Academy of Engineering, he is also the recipient of the Golden Plate Award from the American Academy of Achievement. In addition, he was a dignitary who received the Turing Prize along with his academic colleagues. Understandably, the rejection of his article surprises and stunned many in academia and AI. In a recent Tweeter, the author said that “the problem is not with NeurIPS, but with very selective conference screening practices in emerging areas.”
Likewise, in a post on LinkedIn, he said: “I am happy to report that the article has been rejected from NeurIPS 2021.”
Source: Linkedin publication
As he explained and what seems to be the likely explanation, it was all about doing double-blind reviews. The discussion remains as to whether this was a good thing, although LeCunn expressed it.
Join our Discord server. Be part of an engaging online community. Join here.
Subscribe to our newsletter
Receive the latest updates and relevant offers by sharing your email.