search

JuliaCon / proceedings-review

7 stars 1 forks source link

[PRE REVIEW]: DelayDiffEq: Generating Delay Differential Equation Solvers via Recursive Embedding of Ordinary Differential Equation Solvers #109

Closed whedon closed 1 year ago

whedon commented 1 year ago

Submitting author: @devmotion (David Widmann) Repository: https://github.com/SciML/DelayDiffEq.jl Version: Editor: @odow Reviewers: @Datseris, @ranocha Managing EiC: Valentin Churavy

Author instructions

Thanks for submitting your paper to JCON @devmotion. Currently, there isn't an JCON editor assigned to your paper.

The author's suggestion for the handling editor is @carstenbauer.

@devmotion if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JCON and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JCON submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands
whedon commented 1 year ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 1 year ago 
Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.04 s (1545.2 files/s, 134840.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Julia                           48            808            506           3567
YAML                             6             13             10            172
Markdown                         2             14              0             78
TOML                             3              3              0             45
TeX                              1              1              0             19
-------------------------------------------------------------------------------
SUM:                            60            839            516           3881
-------------------------------------------------------------------------------

Statistical information for the repository 'c6202a796859590274507599' was
gathered on 2022/10/27.
No commited files with the specified extensions were found.
whedon commented 1 year ago

PDF failed to compile for issue #109 with the following error: 

 Can't find any papers to compile :-(
devmotion commented 1 year ago

@whedon generate pdf from branch paper

whedon commented 1 year ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 1 year ago

PDF failed to compile for issue #109 with the following error: 

 /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon/author.rb:72:in `block in build_affiliation_string': Problem with affiliations for Chris Rackauckas\texorpdfstring{~\orcidlink{0000-0001-5850-0663}}{}, perhaps the affiliations index need quoting? (RuntimeError)
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon/author.rb:71:in `each'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon/author.rb:71:in `build_affiliation_string'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon/author.rb:17:in `initialize'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon.rb:205:in `new'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon.rb:205:in `block in parse_authors'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon.rb:202:in `each'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon.rb:202:in `parse_authors'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon.rb:93:in `initialize'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon/processor.rb:38:in `new'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/lib/whedon/processor.rb:38:in `set_paper'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/bin/whedon:58:in `prepare'
    from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
    from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
    from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
    from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
    from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-c5c16aedb3d6/bin/whedon:131:in `<top (required)>'
    from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:in `load'
    from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:in `<main>'
devmotion commented 1 year ago

Strange, it worked locally (and with Github CI) by following the instructions of the template. Possibly a different ruby version? I'll try to fix it in the next few days.

devmotion commented 1 year ago

I'm not sure how to debug and test this issue locally. I can't reproduce it with the JuliaCon instructions and ruby script. I installed ruby 2.6.0 locally and ruby prep.rb works fine (also tried 2.6.6 successfully). So it seems something specific to whedon but I'm not sure how I can run it locally.

devmotion commented 1 year ago

Still haven't managed to reproduce this locally, is there really no simple way to run the build pipeline locally? Debugging with CI seems a bit suboptimal...

However, I found https://github.com/openjournals/whedon/blob/92346a0773a4003bf0ef361b661dc2787f492936/lib/whedon/author.rb#L69-L70 and https://bugs.ruby-lang.org/issues/12451, so maybe that's the underlying issue. I tried to address that, let's see if the paper can be compiled now.

devmotion commented 1 year ago

@whedon generate pdf from branch paper

whedon commented 1 year ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 1 year ago

PDF failed to compile for issue #109 with the following error: 

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
===========Latexmk: Missing input file: 'mathtools.sty' from line
  '! LaTeX Error: File `mathtools.sty' not found.'
Latexmk: Missing input file: 'mathtools.sty' from line
  '! LaTeX Error: File `mathtools.sty' not found.'
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF
devmotion commented 1 year ago

OK, so the first issue is fixed. So what's the common approach here? Adding the LaTeX sty files to the repo? And is there a list with the LaTeX packages that are available when compiling with whedon?

devmotion commented 1 year ago

I found https://github.com/JuliaCon/proceedings-review/issues/79#issuecomment-984731999 - should mathtools be added to the list of available LaTeX packages, similar to https://github.com/openjournals/heroku-buildpack-tex/pull/2, or should it be added to the repo? I'm also not really sure what would have to be added to the repo - it seems there is not mathtools.sty in https://www.ctan.org/tex-archive/macros/latex/contrib/mathtools?

carstenbauer commented 1 year ago

@whedon assign @odow as editor

matbesancon commented 1 year ago

@whedon assign @odow as editor

odow commented 1 year ago

The easiest solution is just to remove the dependence on mathtools. Why is it needed?

devmotion commented 1 year ago

I always use/load mathtools instead of amsmath since it fixes some bugs in amsmath and provides useful additional functionality (see, e.g., https://ftp.acc.umu.se/mirror/CTAN/macros/latex/contrib/mathtools/mathtools.pdf). Here, it seems it is only required for \coloneqq commands. I replaced them with := which hopefully should fix whedon's compilation issues. I think it would be useful nevertheless to support mathtools, as I said it's the package I load for math support in all my LaTeX files and at least I assumed that it is a quite common package.

devmotion commented 1 year ago

@whedon generate pdf from branch paper

whedon commented 1 year ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 1 year ago

PDF failed to compile for issue #109 with the following error: 

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
===========Latexmk: Missing input file: 'orcidlink.sty' from line
  '! LaTeX Error: File `orcidlink.sty' not found.'
Latexmk: Missing input file: 'orcidlink.sty' from line
  '! LaTeX Error: File `orcidlink.sty' not found.'
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF
devmotion commented 1 year ago

Well well, these LaTeX issues are a bit annoying - my feeling is that this could be improved a bit :smile: Even more so since the paper compiles just fine with standard TeXlive installations (as it was the case for the initial submission and all updates) and at least to me it is still unclear what exactly is supported and why (not).

I guess the easiest solution to the latest error is to include the orcidlink.sty file.

devmotion commented 1 year ago

@whedon generate pdf from branch paper

whedon commented 1 year ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 1 year ago

PDF failed to compile for issue #109 with the following error: 

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
===========Latexmk: Missing input file: 'csquotes.sty' from line
  '! LaTeX Error: File `csquotes.sty' not found.'
Latexmk: Missing input file: 'csquotes.sty' from line
  '! LaTeX Error: File `csquotes.sty' not found.'
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF
odow commented 1 year ago

Well well, these LaTeX issues are a bit annoying - my feeling is that this could be improved a bit 😄 Even more so since the paper compiles just fine with standard TeXlive installations (as it was the case for the initial submission and all updates) and at least to me it is still unclear what exactly is supported and why (not).

I guess the easiest solution to the latest error is to include the orcidlink.sty file

Just use the minimal set of packages. Do you really need all the extras?

The bot uses a Docker image with a fixed set of packages. There's essentially zero free time on the editorial team to improve things. @vchuravy was looking at it... and from what I understand, it wasn't an enjoyable experience. If you have time, he can probably point you in the right direction.

devmotion commented 1 year ago

@whedon generate pdf from branch paper

whedon commented 1 year ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 1 year ago

PDF failed to compile for issue #109 with the following error: 

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF
devmotion commented 1 year ago

OK, now I'm a bit lost since it seems the logs don't tell me what the remaining problem(s) are.

devmotion commented 1 year ago

Just use the minimal set of packages. Do you really need all the extras?

In the initial submission I loaded 5 common packages. There were zero LaTeX issues on my computer (and also compiling the LaTeX source code with Github actions worked and works flawlessly), so I did not expect them to cause any problems during the submission. Surely I could try to avoid loading any packages at all (which is not completely possible since plots are generated with pgfplots) but the main point of these packages is that they know better than I how to e.g. fix amsmath and format quotes with references correctly which IMO improves the quality of the LaTeX source code (and often also reduces its size).

To be clear, I don't want to criticize the work of the editors at all - I know how frustrating LaTeX can be and how challenging it can be to try to compile LaTeX files on a new/different computer or with a different TeXlive version. I'm just a bit frustrated that I have to deal with this online debugging here (since I can't reproduce any of the issues locally), and more generally frustrated that one still has to deal with these LaTeX package issues in 2022. Maybe something like tinytex or tectonic could be helpful? But I'm not sure if that would solve the issues with non-default packages at all. Maybe another approach would be to automatically install LaTeX packages in that are listed in some separate text file? But I guess that might open up for different security issues, so maybe that's not a good idea for a public service such as whedon.

odow commented 1 year ago

I only just joined the editorial team, so I've missed the backstory, but from what I understand of my one meeting: everyone agrees that things could/should be better. JOSS have updated to a GitHub action framework to build the PDFs that would give us more control. But updating the JuliaCon stuff is work that no one has the bandwidth to do.

I don't know how the bot works behind the scenes, or how to access the logs, but I wouldn't be surprised if it looks for the paper directory in your repo and copies that, not the entire Git repository. So that may mean it can't find your relative links like ../data and ../src. Could you not just copy those files into the .tex? The source files in particular just rely on a few lines, and they use relative references. So if you edit the text files you'll break the paper. You may as well just copy them into the tex.

I'm just a bit frustrated that I have to deal with this online debugging here (since I can't reproduce any of the issues locally), and more generally frustrated that one still has to deal with these LaTeX package issues in 2022

Do you have a different experience submitting papers to proper Journals??? 😆😆😆

matbesancon commented 1 year ago

One hack that would work is add the sty packages directly in your paper, I think they should be used correctly

devmotion commented 1 year ago

One hack that would work is add the sty packages directly in your paper, I think they should be used correctly

That's how I fixed the orcidlink issue. mathtools and csquotes I removed completely and worked around it in the LaTeX file.

It would be good to be able to inspect the logs before randomly trying more things now. Due to the different (i.e., non-existent) error message I assume the problems with unavailable LaTeX packages might be fixed and it might be a different issue now.

odow commented 1 year ago

It would be good to be able to inspect the logs before randomly trying more things now

I don't know how to inspect the logs, @matbesancon or @vchuravy might know.

But it shouldn't take much trial and error.

There's a few likely candidates:

odow commented 1 year ago

I was wrong on all of the above fronts. After some head bashing to get Whedon to install locally, I found the likely culprit: https://github.com/SciML/DelayDiffEq.jl/pull/255

devmotion commented 1 year ago

Great! And sorry, seems I forgot to reply to your comments above - I was at least certain about the first two not being problematic since I had already fixed the issues with orcidlink (the package was not part of the whedon installation: https://github.com/JuliaCon/proceedings-review/issues/109#issuecomment-1347332598 - adding orcidlink.sty to the repo fixed it) and the affiliations (was a ruby bug: https://github.com/JuliaCon/proceedings-review/issues/109#issuecomment-1320450434).

devmotion commented 1 year ago

@whedon generate pdf from branch paper

whedon commented 1 year ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 1 year ago

PDF failed to compile for issue #109 with the following error: 

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF
devmotion commented 1 year ago

It seems it still doesn't work. But again the error message is not very helpful.

devmotion commented 1 year ago

@whedon generate pdf from branch paper

whedon commented 1 year ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 1 year ago

PDF failed to compile for issue #109 with the following error: 

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF
devmotion commented 1 year ago

@whedon generate pdf from branch paper

whedon commented 1 year ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 1 year ago

PDF failed to compile for issue #109 with the following error: 

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF
devmotion commented 1 year ago

@odow I'm curious, how did you manage to install whedon locally and use it to compile the draft?

I just spent 3 hours trying to set it up locally because I still don't know what the remaining problem is - the PDF compiles fine locally and with a completely standard pdflatex GH action in our repo (as it has always done), the comments above are utterly useless, and the output of https://whedon.theoj.org is only marginally better but still undecipherable:

Your paper failed to compile with the following errors:

Looks like we failed to compile the PDF with the following error: Error producing PDF. ! LaTeX Error: Can be used only in preamble. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.313 \documentclass

I'm very familiar with LaTeX and happy to fix all kinds of issues but it's just not possible if I can't reproduce them and don't have access to any logs.

But it shouldn't take much trial and error.

Well it does. As you can see from the comments above and the commit history (https://github.com/SciML/DelayDiffEq.jl/commits/paper) the only possible strategy so far is to more or less randomly try to change some things. By now I have spent almost as much time on trying to figure out what the problem with whedon is and why it fails than writing the paper. That's insane. Even if the LaTeX system can't be improved for whatever reasons and I have to use old package versions, remove packages, or include sty files (if possible, it won't be feasible for big packages) - could we just get access to proper logs? Or some clear and simple instructions for how to install and run whedon locally? Whatever I try, ruby package installation always fails, as soon as I have worked around some issue, a new one pops up.

devmotion commented 1 year ago

FYI even though I had already tried them without success, I tried your suggestions above once more. It still fails with the same error message.

devmotion commented 1 year ago

@whedon generate pdf from branch paper