setup.py: Copy NenuFAR configurations on install

David-McKenna commented 3 years ago

Hey Tobia,

Another quick PR for you. I was installing dreamBeam on our compute nodes and I was running into an error when trying to use on_pointing_axis_tracking. Even though I was using a LOFAR station, I was getting an error about NenuFAR files being missing:

    stnfeed = load_mountedfeed(telescopename, stnid, band, antmodel)
  File "/usr/local/lib/python3.6/dist-packages/dreambeam/telescopes/rt.py", line 42, in load_mountedfeed
    telescope_plugins = get_tel_plugins()
  File "/usr/local/lib/python3.6/dist-packages/dreambeam/telescopes/rt.py", line 31, in get_tel_plugins
    pluginpath)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.6/dist-packages/dreambeam/telescopes/NenuFAR/_telescope.py", line 15, in <module>
    telhelper = TelescopePlugin(TELESCOPE_NAME, MountedFeedFixed, POLCRDROT)
  File "/usr/local/lib/python3.6/dist-packages/dreambeam/telescopes/rt.py", line 57, in __init__
    self.bands = self.feedplugin.get_bands()
  File "/usr/local/lib/python3.6/dist-packages/dreambeam/feeds/feedplugins.py", line 140, in get_bands
    coeffilemetas = self._search_coeff_files()
  File "/usr/local/lib/python3.6/dist-packages/dreambeam/feeds/feedplugins.py", line 129, in _search_coeff_files
    ls = os.listdir(modelspecdir)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.6/dist-packages/dreambeam/telescopes/NenuFAR/share'

I checked out setup.py and it looks like the only the LOFAR configuration files were being copied, this PR just adds in the NenuFAR files as well and resolves the issue I was having above.

Cheers, David

2baOrNot2ba commented 3 years ago

Thanks David. Ultimately want a genuine plugin for each telescope, but for now needs your fix.

David-McKenna commented 3 years ago

Hey Tobia,

Would adding something like this to setup.py be a better stop gap in the meantime then?

...
import os
install_folder = os.path.dirname(os.path.abspath(__file__))
scopes_folder = os.path.join(os.path.join(install_folder, 'dreambeam'), 'telescopes')
telescopes = [scope for scope in os.listdir(scopes_folder) if os.path.isdir(scope)]
standard_includes = ['share/*.cc', 'share/simmos/*.cfg', 'share/alignment/*.txt']

package_data = {'dreambeam': ['configs/*.txt']}
for scope in telescopes:
      package_data['dreambeam.telescopes.' + scope] = standard_includes

setup(name='dreamBeam',
...
      package_data=package_data,
...

One other quick thing, do you still have functions for calculating beam pointing responses of the sky in dreamBeam? Eg figures 10/11 in the beamModelTester paper ( https://arxiv.org/pdf/1908.02141.pdf ) or slides 11/12 of the presentation at the ILT 2017 meeting ( https://www.astron.nl/ilttom2017/Documents/ILTTOM2017_2_SE607_1.pdf ), I've had a quick scan both dreamBeam and AntPat but I don't think I've come across functions that would generate outputs like that.

Cheers

2baOrNot2ba commented 3 years ago

Yeah, good idea. At least this latest snippet avoids explicit mention of telescope plugins...

2baOrNot2ba commented 3 years ago

Regarding beamModelTester, @creaneroDIAS based that paper on beamWrapper and beamModelTester. He applied it to LOFAR ACC (vis cubes) files from the IE and SE stations, and had been converted to BST-like (beam powers) HDF5 files using an iLiSA app, I should probably upload, at some point. I also have an app to plot light curves from both Db model and obs, directly from iLiSA, that I again should upload. (Let me know if your interested...) If you want to correct for beam model in images and get proper IAU coordinates, I have introduced that feature in the ilisa_image CLI a few months ago. It takes LOFAR XST or ACC as input. I am also working on beam model correction in Helio coordinates for LOFAR4SW... Do you have any interest in this, David?

David-McKenna commented 3 years ago

Hey Tobia,

Ah, I think I mistintrepted what those figures were -- I assumed they were a predicted beam response across the the entire sky, given the station and it's ra/dec pointing, rather than images generated from ACCs/XSTs.

I was looking for a shortcut to generate those beam responses as since we're looking at a lot of transients at the moment, I intend to try get a better estimate Tsky for a given beam pointing and frequency by including the sidelobes and seeing if that gives a result that gives a more stable/accurate result, considering it's quite obvious when A-team member transit the sidelobes but the standard method for calculating Tsky from the main beam can't adjust our predicted sensitivity to account for that. I was wondering if dreamBeam had a component to perform that with some corrections for the beam to get a better estimate of the sky response rather than just doing an FFT over the tile/antenna locations.

As for what I'm currently doing with dreamBeam, it was a random idea I had a while back to try apply a rough phase/polsarization correction from the Jones matrices you generate for a given pointing to our CEP packet data to try improve polarized measurements and maybe supress the RMS noise a bit if we're lucky.

I got an implementation working over the weekend but haven't had much time to test it as I've got several other things to attend to this week, but at first glance it at least modified Stokes V emission from being negative emission to positive emission, with a slight reduction (~0.5%) in RMS noise on Stokes I for a high altitude observation with a very minimal performance penalty. A rough outline of the methodology I was trying is

I use a script to generate Jones matrices from dreamBeam, generate the inverted Jones matrix and then pass them to my voltages library
I then apply them to the voltages for each frequency here and here, updating the Jones matrix every time step (~1-5s depedinng on user input)
Those voltages are then passed on to whatever requested, whether I'm just outputing the voltages, forming a Stokes parameter or using them for coherent dedispersion

I probably have some gap in my knowledge that makes it so that applying a correction like that won't have any profound effect on the data or further corrupts it given I wasn't able to find anyone doig something like this for beamformed data outside of the imaging attempts with the core (where it's needed to account for the different stations anyway), but I thought I'd just give it a shot and see what happens.

The helio coorinates could be handy though, in the script above I have it so that I recalculate the ra/dec if the given pointing is not in J2000 since we have people observing the Sun, Jupiter and uranus with the station, so a way to accept any of the caracore basis' would be handy.

Cheers, David

2baOrNot2ba commented 3 years ago

David, seems like we should have a chat. I've got too much to add and it won't suit GH. Can I contact you tomorrow?

David-McKenna commented 3 years ago

Could we say Thursday? Tomorrow is going to be fairly hectic for me between a meeting with my supervisors for most of the morning and handling the observations we're recording during local mode aftwards

2baOrNot2ba / dreamBeam

setup.py: Copy NenuFAR configurations on install #7