search

epic-astronomy / LWA_EPIC

The LWA-specific implementation of the EPIC correlator
http://livetv.epic-astronomy.org
MIT License
3 stars 3 forks source link

New LSL API #12

Closed jaycedowell closed 4 years ago

jaycedowell commented 4 years ago

Support for the LSL v2.0 API.

jaycedowell commented 4 years ago

I think that only touches the LSL stuff.

adampbeardsley commented 4 years ago

I did a couple quick validation tests. Using master and this branch, I ran:

python LWA_bifrost.py --offline --tbnfile=/data5/LWA_SV_data/data_raw/TBN/Jupiter/058161_000086727 --imagesize 64 --imageres 1.79057 --nts 512 --channels 4 --accumulate 50 --ints_per_file 40

Which resulted in images that look like this: image (which is which doesn't matter because the images are pixel-by-pixel identical)

I also ran:

python LWA_bifrost_DFT.py --offline --tbnfile=/data5/LWA_SV_data/data_raw/TBN/Jupiter/058161_000086727 --imagesize 64 --imageres 1.79057 --nts 512 --channels 4 --accumulate 50 --ints_per_file 40

which also resulted in identical images.

So I'm happy.

This will be a breaking change -- Anyone using master (or merges master into their own branch) will need to update LSL. I think @HaraKrish is the main user right now. Are you good with this Hari?

HaraKrish commented 4 years ago

I have already used them and verified that the output images are correct, no issues. However, I noticed when we set the flag "--profile" to display the timing. The times are atleast 50 % higher than what we get with the earlier version. I'm not sure if it will have any effect on pipeline performance . But just wanted to make a note of it.

adampbeardsley commented 4 years ago

Hmm, that does sound worrisome. Does --profile give more of a breakdown so we can try to narrow down what causes the increased time?

HaraKrish commented 4 years ago

No it does not give a break-up but the flag "--profile" is for timing specific CPU parts of the pipeline using the CPU clock. I will make a quantitative comparison with exact numbers wherever we display the times before getting on further with this. Just to clarify, this is independent of the GPU part of the code which is unaffected.

jaycedowell commented 4 years ago

I wouldn't think that the LSL API change would matter for the timing. The LSL related code is only hit once per sequence when the delays/phases are updated.

adampbeardsley commented 4 years ago

It seems Hari's benchmarking is a bit funny, and probably due to some running problems (see slack conversation from today). I was able to benchmark and saw no significant difference between the branches.