search

genenetwork / genenetwork2

GeneNetwork (2nd generation)
http://gn2.genenetwork.org/
GNU Affero General Public License v3.0
34 stars 24 forks source link

metadata &textfile generator script #777

Open Alexanderlacuna opened 1 year ago

Alexanderlacuna commented 1 year ago

Description

How should this be tested?

Any background context you want to provide?

What are the relevant pivotal tracker stories?

Screenshots (if appropriate)

Questions

Alexanderlacuna commented 1 year ago

@BonfaceKilz @zsloan could you help in reviewing this

Alexanderlacuna commented 1 year ago

you can use this branch for testing:https://github.com/genenetwork/genenetwork2/pull/780

zsloan commented 1 year ago

I'm testing this now (btw sorry about the delay on that - in the future feel free to just bug me over e-mail about stuff like this), and I think it's working correctly because it's been running a long time without errors.

How do I test that it's run correctly? It's still running, but the directory I think it's writing to (in this case /tmp/Probesets) hasn't been updated since soon after I ran the script (which might be normal, but not sure).

zsloan commented 1 year ago

After running for a while, I got the following error (which seems to be due to hitting some size limit and is likely due to the LMDB installation on Tux01 or something rather than anything in the script): lmdb.MapFullError: mdb_put: MDB_MAP_FULL: Environment mapsize limit reached

Alexanderlacuna commented 1 year ago

After running for a while, I got the following error (which seems to be due to hitting some size limit and is likely due to the LMDB installation on Tux01 or something rather than anything in the script): lmdb.MapFullError: mdb_put: MDB_MAP_FULL: Environment mapsize limit reached

it has to do with allocating size of lmdb I will fix this ....Hope this is not occurring for --metadata they are quite small.

zsloan commented 1 year ago

Ah thanks, I'll try it again with that latest commit.