sagemath / trac-to-github

Script to migrate Trac tickets to GitHub issues and the Trac wiki to markdown. Input: https://trac.sagemath.org/ ➠ Intermediate: https://github.com/sagemath/trac_to_gh ➠ Output: https://github.com/sagemath/sage/issues
https://trac.sagemath.org/ticket/30363
7 stars 5 forks source link

Fix up handling of repository_files / attachments for some types #176

Closed mkoeppe closed 1 year ago

mkoeppe commented 1 year ago

There are a few files that are causing errors on import into github.com

[["https://github.com/sagemath/sage/files/ticket870/GNUmakefile.in", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket954/test_gcc_version.pl", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket1754/firefox_gnutls_1.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket1754/firefox_gnutls_2.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket1754/firefox_gnutls_3.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket2100/Before.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket2100/After.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket2628/literate_notebook.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket2754/plot-sin.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket2770/plot-region.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket2900/foo.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket3210/Z5.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket3321/sage0.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket3321/sage0.2.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket3541/trac_3541-multiedge-graph-example-plot.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket3599/slider-bg-3.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket3713/8bbd3a5a829588ba2782209a6dda0314.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket3886/sage0.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket3886/sage1.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4194/foo.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4384/plot.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4462/circle_sage3.1.1.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4462/circle_sage3.1.4.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4529/sage0.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4530/sage0.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4575/html_table.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4575/html_table1.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4878/4878_example.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4878/trac_4878-sample-plot-bw.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket4976/fill1.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5365/t2.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5366/t2.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5366/fixed.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5511/mesh_function.jpeg", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5532/65eaf71bcdf76e690e357de43d619c99.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5564/trac_5564-problem.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5601/sage_colors.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5606/sqrt.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5606/zeta.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5610/list-function-pdf.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5645/plot.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5668/nb_mem.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5668/8bbd3a5a829588ba2782209a6dda0314.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5864/eclib-20080310.p7.patch.gz", "Skipped import: repository file upload failed"],
 ["https://github.com/sagemath/sage/files/ticket5916/mathematica.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket5975/petersen-latex.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6001/tearout_opera.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6001/tornout_firefox.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6012/graph.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6098/curve.jpg", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6238/is_planar-example.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6440/serve.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6747/tree2.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6747/tree.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6864/sage.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6864/sage0.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6864/sage2.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6864/0.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6864/zz.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6872/trac_6872-without-patch.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6875/polar_1.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket6875/polar_2.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket7003/jmol-lighting-screenshot.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket7299/Triangle-sage_4.1.1.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket7299/Triangle-Sage_4.1.2.png", "Skipped import: invalid or missing repository file"],
 ["https://github.com/sagemath/sage/files/ticket7365/tmp_4.png", "Skipped import: invalid or missing repository file"],

The first two should be compressed:

           INFO     Attachment filename='GNUmakefile.in' mimetype='text/plain'                                                                                                             migrate.py:1717
           INFO     Attachment link https://github.com/sagemath/sage/files/ticket870/GNUmakefile.in                                                                                        migrate.py:1756

And the png files should be attachments instead. - per github contact.

kwankyu commented 1 year ago

Which step of the migration process do we fix and start again?

mkoeppe commented 1 year ago

Running migrate.py again after fixing the code to make a fresh migration archive. I've already committed a change in the output format (batching json), which our github contact suggested to give a much(?) faster import speed.

kwankyu commented 1 year ago

OK. Thanks.

mkoeppe commented 1 year ago

Done in 830c4f3...bbb7375

mkoeppe commented 1 year ago

Also https://github.com/sagemath/sage/files/ticket9385/dMMCACHEEDGE.LOG is rejected.

kwankyu commented 1 year ago

I don't see anything special about it. It is just a plain text file.

mkoeppe commented 1 year ago

Fixed by c79cdb7