Generalize paper with code to other sciences, not only AI

[LifeIsStrange]

Hi! First of all I would like to say thank you for making the world a far better place. I have tears in the eyes contemplating the positive impact that your "simple website" can have on science and on scientists.

I had the idea, many years ago to make a website that centralize state of the art performance and give it transparency, visibility! But I didn't make it, you did, and I'm happy for you. Such a website, once it becomes mainstream can be transformative for science, because it make scientists in competition for state-of-the-art, while before they compared against "an opaque, old sota baseline" and finding the SOTA was a quest, people "guessed" this was the SOTA. Paperswithcode.com show it and save much time. More than that, you easily allow scientists to read the direct competition and to inspire from it (maybe you could incentivise collaboration and reuse?) But this is already a massive improvement.

But please don't just save AI research! Save all scientists! Paper with code should have a category per science and applied science. Not just AI but Algorithms per complexity and constant cost, databases performance, psychology papers and Empirical benchmarked effectiveness (the website would benefit to see reproduced results (but for AI too)), solar energy performance, Empirical political experiments, medecine per medicament or combinaison of medicament effectiveness to cure from X disease, etc the possibilities are fascinating. So yes it would not always be papers "with code" but it would highlight reproducible, reproduced papers which is what really matters, code is contingent.

Please save science by increasing sota visibility, learning from others, collaborations and science epistemology (reproducibility, double blind, sample size visibility, etc)

Maybe you could send a letter to a rich philanthrope explaining him how you can make earth a better place. Maybe you can ask funding from https://www.effectivealtruism.org/ or https://80000hours.org/

[LifeIsStrange]

I believe that for others sciences without code, your marketing has a big potential of playing with the sense of code, via an equivocation. That is https://en.m.wikipedia.org/wiki/Academic_honor_code So for other sciences, a paper with code would be a paper with an ethical / honor code, which means being scientifically rigorous, having a strong epistemology.

[VladimirAlexiev]

@LifeIsStrange PWC covers more sciences, see https://portal.paperswithcode.com/:

• Machine Learning 114,210 Papers with Code • 11,894 Benchmarks • 4,571 Tasks • 15,536 Datasets
• Computer Science 12,970 Papers with Code
• Physics 8,723 Papers with Code
• Mathematics 4,782 Papers with Code
• Astronomy 4,769 Papers with Code
• Statistics 3,477 Papers with Code

[LifeIsStrange]

Hi @VladimirAlexiev happy to see someone working on ontologies/semantics I am personally a fan of https://semanticsarchive.net/cgi-bin/browse.pl (computational linguistics)

Anyway, about your comment, yes but no.. They do cover multiple sciences (IIRC wasn't the case in 2019?) BUT they still managed to utterly fail by removing the main revolutionary feature, which is not of giving access to code, but of listing datasets and associated SOTA leaderboards https://paperswithcode.com/sota is only about machine learning and the day they will change that, science will progress much faster and progress itself will become much more discoverable. for example the concept exist for covid https://c19early.org/ but is otherwise extremely rare which is tragic, stagnant, and obscurantist.