search

transientskp / tkp

A transients-discovery pipeline for astronomical image-based surveys
http://docs.transientskp.org/
BSD 2-Clause "Simplified" License
19 stars 15 forks source link

regression problem, tkp 4.0 might be slower than tkp 3 #544

Closed gijzelaerr closed 7 years ago

gijzelaerr commented 7 years ago

@mkuiack please post links to the profiles and logs here.

mkuiack commented 7 years ago

The TraP 4.0 project directory ais001:/home/kuiack/mkuiack/speedtest contains profiles and logs. The banana sites for the databases http://banana.transientskp.org/r4/vlo_speedtest/ and http://banana.transientskp.org/r4/vlo_speed_40/ are also related.

If you want to just start over we can do that. There are lots of images with various subband integrations in ais001:/mnt/nex*/data/images

gijzelaerr commented 7 years ago

did you give banana access to the database?

mkuiack commented 7 years ago

I'm not sure why I didn't before, but I did now, and it works.

gijzelaerr commented 7 years ago

conclusion is, we just try to do much in one go with all the varmetric updating. Impossible to parallelise since everything is done in the database. I think this is a dead end, the next TraP should move away from the design of moving all calculations to the database and just use it for storing lightcurves.