quatrope / ProperImage

Proper Image utils for astronomy.
BSD 3-Clause "New" or "Revised" License
26 stars 6 forks source link

Update the used module numpydb to `diskcache` for stamp storage #42

Open BrunoSanchez opened 4 years ago

BrunoSanchez commented 4 years ago

There is an implementation for a numpy array storage, that uses pickle. This is critical so we don't keep in memory stamps of every star. The current implementation though, requires the creation of files, which for many reasons can persist later as garbage in the local directory (see #11).

Using diskcache library could avoid such issue, and clean the base code of additional modules.

BrunoSanchez commented 4 years ago

A comment on numpyDB: the code is public in two places

Maybe it makes sense to make scitools a dependency, it is in PyPI and we could avoid mantaining numpyDB.

martinberoiz commented 4 years ago

Yes, judging by the last commit (https://github.com/esheldon/numpydb/commit/5ccce22913e4b745d21c4d68fb239c365e926bee) and lack of versions in esheldon's repo, I would go with scitools copy.

leliel12 commented 4 years ago

+1

El mar., 13 oct. 2020 a las 12:11, Martin Beroiz (notifications@github.com) escribió:

Yes, judging by the last commit (esheldon/numpydb@5ccce22 https://github.com/esheldon/numpydb/commit/5ccce22913e4b745d21c4d68fb239c365e926bee) and lack of versions in esheldon's repo, I would go with scitools copy.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/toros-astro/ProperImage/issues/42#issuecomment-707808319, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACJAAWQRWEO3DMESAVBRWDSKRU2PANCNFSM4QMQBFKA .

-- Juan B Cabral

leliel12 commented 4 years ago

Mey be we can check what library is more used

El mar., 13 oct. 2020 a las 12:46, Juan BC (jbc.develop@gmail.com) escribió:

+1

El mar., 13 oct. 2020 a las 12:11, Martin Beroiz (< notifications@github.com>) escribió:

Yes, judging by the last commit (esheldon/numpydb@5ccce22 https://github.com/esheldon/numpydb/commit/5ccce22913e4b745d21c4d68fb239c365e926bee) and lack of versions in esheldon's repo, I would go with scitools copy.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/toros-astro/ProperImage/issues/42#issuecomment-707808319, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACJAAWQRWEO3DMESAVBRWDSKRU2PANCNFSM4QMQBFKA .

-- Juan B Cabral

-- Juan B Cabral

BrunoSanchez commented 4 years ago

Scitools is a big library and last commit is from 2016. I don't know if it supports Python > 2.6

esheldon's repo is only about NupyDB, latest commit is from 2019 and there is some extra functionality which I don't use. However he mentions some seg fault in his latest commit.