Commit expression collections

[StevenCannon-USDA]

I notice that several expression collections have changes in the Data Store but haven't been committed to the repository for some time. Are these in a state where they can be added to datastore-metadata? @adf-ncgr @el239

[adf-ncgr]

@StevenCannon-USDA yes this is probably a good time to do that, esp. given that the autocontent browser config will want to use those CHECKSUM files to determine the lists of bigwigs; @el239 I may or may not have given you instructions covering this aspect of datastore maintenance. If not, let me know and @ctcncgr or I can cover it with you.

[el239]

@adf-ncgr No, I don't yet know how to do this. So far I've been fabricating checksums locally to test bigwigs with the autocontent.

[adf-ncgr]

OK, I'll add some instructions to the datastore-metadata README, which seems a little in need of refresh anyway (doesn't currently mention the CHECKSUM files for example).

[adf-ncgr]

@el239 assigning this one to you, although the README updates are still something of a work in progress.

[el239]

Okay! I completed my README review with a comment. I was able to successfully push the .md5 and yaml for a new Arahy dataset. I'll upload the rest after checking this works with the autocontent, then close the issue when everything is up-to-date.

[StevenCannon-USDA]

I am running into this conflict now when I attempt git pull:

error: The following untracked working tree files would be overwritten by merge:
    Glycine/max/expression/Wm82.gnm6.ann1.expr.Magellan.Pelaez-Vico_Sinha_2023/README.Wm82.gnm6.ann1.expr.Magellan.Pelaez-Vico_Sinha_2023.yml
    Glycine/max/expression/Wm82.gnm6.ann1.expr.Wm82.Sreedasyam_Plott_2023/CHECKSUM.Wm82.gnm6.ann1.expr.Wm82.Sreedasyam_Plott_2023.md5
    Glycine/max/expression/Wm82.gnm6.ann1.expr.Wm82.Sreedasyam_Plott_2023/README.Wm82.gnm6.ann1.expr.Wm82.Sreedasyam_Plott_2023.yml
    Glycine/max/expression/Wm82.gnm6.ann1.expr.mixed.Kour_Boone_2014/CHECKSUM.Wm82.gnm6.ann1.expr.mixed.Kour_Boone_2014.md5
    Phaseolus/vulgaris/expression/G19833.gnm2.ann1.expr.Negro_jamapa.ORourke_Iniguez_2014/README.G19833.gnm2.ann1.expr.Negro_jamapa.ORourke_Iniguez_2014.yml
Please move or remove them before you merge.

I moved those five files into my home directory, then did git pull. @el239: if you need those files, let me know and I'll transfer them back to you.

There are still a number of expression collections that haven't been committed. It would be good to do that, in order to avoid collisions of the type above.

[el239]

Thanks, though I'm still seeing these files in main, feel free to delete those in your home dir.

Are you referring to the README/.yaml metadata of expression collections? I believe they're all committed, please let me know if something is missing. I do need to update the checksums to reflect the addition of bigwigs in the datastore, but I don't notice anything out of order apart from that.

[StevenCannon-USDA]

I am seeing the uncommitted files in our instance of the Data Store -- on peanutbase-stage.usda.iastate.edu:

peanutbase-stage:/usr/local/www/data/v2 $ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    deleted:    Glycine/max/expression/Wm82.gnm6.ann1.expr.mixed.Kour_Boone_2014/README.glyma.Wm82.gnm6.ann1.expr.mixed.Kour_Boone_2014.yml

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    Glycine/max/expression/Wm82.gnm6.ann1.expr.Wm82.Libault_Farmer_2010/
    Glycine/max/expression/Wm82.gnm6.ann1.expr.mixed.Kour_Boone_2014/README.Wm82.gnm6.ann1.expr.mixed.Kour_Boone_2014.yml
    Glycine/max/expression/Wm82.gnm6.ann1.expr.mixed.Kour_Boone_2014/README.genotype
    Vigna/unguiculata/expression/IT97K-499-35.gnm1.ann1.expr.IT97K-499-35.Yao_Jiang_2016/

[el239]

Thank you for the elaboration, I had a misconception about the git paradigm being used. I think I've resolved the issue, and also have updated the CHECKSUM files.

[StevenCannon-USDA]

@el239 That looks much better. nothing to commit, working tree clean. Thanks for having another look.