0xCoto / Virgo

📡 Virgo: A Versatile Spectrometer for Radio Astronomy
https://virgo.readthedocs.io/en/latest/
GNU General Public License v3.0
141 stars 23 forks source link

Useless spectra and wola.grc does not open #5

Open iHD992 opened 3 years ago

iHD992 commented 3 years ago

Hi 0xCoto, probably not real issues but wrong handling by myself:

I am building a radio telescope. Until my RTL-SDR.com V3 arrives and my telescope is ready, I play around with an old DVB-T stick. Hardware: Simple telescopic dipole antenna, RTL2832U + FC0013, Raspberry Pi 4 Software: Latest Raspbian and GNU Radio Compnion 3.7.13.4; Atro-Virgo installed using pip. The hardware setup works as I can see nice waterfall spectra with GNU radio (RTL-SDR source plus QT waterfall sink) and RF Gain == 1.

I just wanted to aquire some basic data in the range of FM radio to have something to play with. Here is my code: `import virgo

counter = 8752e4 while counter < 11000e4:

# Define observation parameters
observation = {
    'dev_args': '',
    'rf_gain': 1,
    'if_gain': 20,
    'bb_gain': 20,
    'frequency': counter,
    'bandwidth': 256e4,
    'channels': 64,
    't_sample': 5,
    'duration': 15
}

# Data acquisition
virgo.observe(obs_parameters=observation, obs_file=str(counter), start_in=0)

import numpy

numpy.savetxt(str(counter)+".csv", numpy.fromfile(str(counter), dtype='float32'), delimiter=",")
counter += 256e4

` It generates 9 files with 3 spectra each. Here the plotted results: (https://ibb.co/87hwqKL) I can just see artifacts. No idea why the first four and the latter fife are different. As FM radio is so strong, I would have expected to clearly see the fm stations. Do you have an idea what I am doing wrong?

I tried to open wola.grc to analyze the problem but gnu companion just gave me some errors. What is your version of GNU Companion?

0xCoto commented 3 years ago

What happens if you try plotting the data with Virgo? Also, I'd increase the gain to 30 dB for RF, IF and BB to help with sensitivity:

import virgo

# Define observation parameters
obs = {
    'dev_args': '',
    'rf_gain': 30,
    'if_gain': 30,
    'bb_gain': 30,
    'frequency': 90e6,
    'bandwidth': 2.4e6,
    'channels': 2048,
    't_sample': 0.1,
    'duration': 10
}

# Begin data acquisition
virgo.observe(obs_parameters=obs, obs_file='observation.dat')

# Plot data
virgo.plot(obs_parameters=obs, obs_file='observation.dat', plot_file='plot.png')

By the way Virgo has an integrated function built into the package to survey wideband spectra:

# Monitors RFI. f_lo and f_hi define the frequency limits in Hz and data is the directory in which the .dat RFI survey files are stored in.
monitor_rfi(f_lo, f_hi, obs_parameters, data='rfi_data')

# Plots RFI data. rfi_parameters should be the same as obs_parameters, but should also include 'f_lo': f_lo. Set dB=True for a wider dynamic range.
plot_rfi(rfi_parameters, data='rfi_data', dB=True, plot_file='plot.png')

But see if you can detect FM stations with the original script first to verify your system is working properly.

Also, what error are you getting when you try opening wola.grc with GNU Radio Companion? It's been extensively tested with v3.7.

iHD992 commented 3 years ago

Tanks for the quick reply.

Slowly adapted your code and tested it. Everything is fine an I could not reproduce the problem with the spectra.

These is a log from a failed attempt to open wola.grc: ` Loading: "/home/pi/wola.grc" Error: /home/pi/wola.grc:68:90:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-pjax-transient /home/pi/wola.grc:75:69:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-pjax-transient /home/pi/wola.grc:119:112:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-pjax-transient /home/pi/wola.grc:138:10:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: link line 136 and head /home/pi/wola.grc:168:27:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'nbsp' not defined /home/pi/wola.grc:195:267:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'rarr' not defined /home/pi/wola.grc:209:332:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'rarr' not defined /home/pi/wola.grc:210:308:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'rarr' not defined /home/pi/wola.grc:233:299:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'rarr' not defined /home/pi/wola.grc:270:262:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'rarr' not defined /home/pi/wola.grc:278:312:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'rarr' not defined /home/pi/wola.grc:279:335:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'rarr' not defined /home/pi/wola.grc:297:52:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: div line 296 and option /home/pi/wola.grc:297:59:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: div line 289 and form /home/pi/wola.grc:351:7:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: img line 332 and a /home/pi/wola.grc:352:6:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: a line 325 and li /home/pi/wola.grc:354:6:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: li line 324 and ul /home/pi/wola.grc:392:7:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: img line 373 and a /home/pi/wola.grc:393:6:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: a line 366 and li /home/pi/wola.grc:424:7:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: img line 405 and a /home/pi/wola.grc:425:6:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: a line 398 and li /home/pi/wola.grc:428:6:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: li line 397 and ul /home/pi/wola.grc:430:19:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: li line 365 and div /home/pi/wola.grc:431:15:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: ul line 362 and label /home/pi/wola.grc:432:8:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: ul line 321 and form /home/pi/wola.grc:433:7:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: img line 317 and div /home/pi/wola.grc:435:17:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: input line 316 and div /home/pi/wola.grc:441:21:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'nbsp' not defined /home/pi/wola.grc:447:25:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'nbsp' not defined /home/pi/wola.grc:449:13:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: input line 299 and div /home/pi/wola.grc:450:11:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: label line 298 and div /home/pi/wola.grc:451:9:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: form line 297 and div /home/pi/wola.grc:452:10:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: div line 288 and header /home/pi/wola.grc:462:28:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-pjax-replace /home/pi/wola.grc:462:28:FATAL:PARSER:ERR_SPACE_REQUIRED: attributes construct error /home/pi/wola.grc:462:28:FATAL:PARSER:ERR_GT_REQUIRED: Couldn't find end of Start Tag div line 462 /home/pi/wola.grc:490:5:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-commit-hovercards-enabled /home/pi/wola.grc:490:5:FATAL:PARSER:ERR_SPACE_REQUIRED: attributes construct error /home/pi/wola.grc:490:5:FATAL:PARSER:ERR_GT_REQUIRED: Couldn't find end of Start Tag div line 487 /home/pi/wola.grc:493:24:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute itemscope /home/pi/wola.grc:493:24:FATAL:PARSER:ERR_SPACE_REQUIRED: attributes construct error /home/pi/wola.grc:493:24:FATAL:PARSER:ERR_GT_REQUIRED: Couldn't find end of Start Tag div line 493 /home/pi/wola.grc:494:59:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-pjax-container /home/pi/wola.grc:637:55:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:641:57:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:645:64:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:649:58:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:653:59:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:657:59:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:661:59:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:694:57:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-menu-button /home/pi/wola.grc:698:160:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute preload /home/pi/wola.grc:715:13:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-pjax /home/pi/wola.grc:715:13:FATAL:PARSER:ERR_SPACE_REQUIRED: attributes construct error /home/pi/wola.grc:715:13:FATAL:PARSER:ERR_GT_REQUIRED: Couldn't find end of Start Tag a line 712 /home/pi/wola.grc:717:11:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: div line 686 and a /home/pi/wola.grc:752:65:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'nbsp' not defined /home/pi/wola.grc:756:60:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'nbsp' not defined /home/pi/wola.grc:809:100:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-paste-markdown-skip /home/pi/wola.grc:13180:54:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: details-dialog line 13179 and option /home/pi/wola.grc:13180:61:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: details line 13177 and form /home/pi/wola.grc:13181:134:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'hellip' not defined /home/pi/wola.grc:13181:171:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute autofocus /home/pi/wola.grc:13182:60:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-close-dialog /home/pi/wola.grc:13183:8:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: input line 13181 and form /home/pi/wola.grc:13183:29:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: form line 13180 and details-dialog /home/pi/wola.grc:13184:14:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: div line 674 and details /home/pi/wola.grc:13189:9:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: main line 494 and div /home/pi/wola.grc:13192:12:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: header line 151 and main /home/pi/wola.grc:13195:9:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: body line 140 and div /home/pi/wola.grc:13201:38:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'copy' not defined /home/pi/wola.grc:13240:5:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:13242:58:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:13243:59:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute hidden /home/pi/wola.grc:13246:108:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute open /home/pi/wola.grc:13249:144:FATAL:PARSER:ERR_ATTRIBUTE_WITHOUT_VALUE: Specification mandate value for attribute data-close-dialog /home/pi/wola.grc:13263:10:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: meta line 133 and body /home/pi/wola.grc:13264:8:FATAL:PARSER:ERR_TAG_NAME_MISMATCH: Opening and ending tag mismatch: meta line 132 and html /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 130 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 129 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 128 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 126 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 124 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 122 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 119 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 113 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 111 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 108 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 106 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 103 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 100 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 99 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 80 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 79 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 78 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 77 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 75 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 68 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 63 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 54 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 53 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 52 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 51 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 48 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 17 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 16 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 15 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 14 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 13 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 12 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag link line 11 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag meta line 10 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag head line 9 /home/pi/wola.grc:13266:1:FATAL:PARSER:ERR_TAG_NOT_FINISHED: Premature end of data in tag html line 8

Failure`

0xCoto commented 3 years ago

This is odd. How did you install Virgo? Did you simply pip install astro-virgo?

iHD992 commented 3 years ago

Yes. Does this file need additional files to be in the same directory or is it self suficient?

0xCoto commented 3 years ago

No, it appears you've installed the package correctly. You should be able to work with the module fine since the .py files (compiled GNU Radio flowcharts) work fine for you.

What I find odd is the .grc error log. For example, I'm not sure how your GNU Radio manages to pick up 'rarr' here:

/home/pi/wola.grc:233:299:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'rarr' not defined

I suspect there might be some kind of mismatch between the Git file and the installed version on your system. Could you please share the wola.grc file here for me to check and compare with the original (working version) wola.grc flowchart? Although a more probable idea is you're using a different (more up-to-date?) version than the GNU Radio version the flowchart was created with. Updating to GR 3.8/3.9 is definitely prioritized on my to-do list and should prevent such problems, although many users still appear to use GR 3.7.