fivpt reports exactly 100 for Tsys with non-negative Tcal in 9.13.2 for continuous cal

[wehimwich]

On May 11, 2021, at 9:31 PM, Stuart Weston <@stuartdweston> wrote:

What I can't work out is why it comes up with a Tsys of 100.000 :

021.131.23:05:42.88#fivpt#tsys 71.274 32.456 100.000 0.1592

In the past I have managed to get a value in there, can't remember what I did as it was some years ago. But now with my current c.rxg file setup it always put's in 100.

[wehimwich]

@stuartdweston wrote (from #122):

Looking at the code in incom/get_rxgain_files.c I see a commented out line to print the file names, so I uncommented and did a make

When starting the FS I now see it reads the correct rxg files :

running incom file /usr2/control/rxg_files/cl.rxg file /usr2/control/rxg_files/c.rxg file /usr2/control/rxg_files/l.rxg file /usr2/control/rxg_files/x.rxg

But doesn't explain the Tsys 100 in fivpt, as in the "c.rxg" I have not used -100 (see earlier email and response to issue).

[wehimwich]

On May 17, 2021, at 5:07 PM, Stuart Weston <@stuartdweston> wrote:

Please find attached the c.rxg file, you will see TCAL is not -100.

When I use onoff with any source that line in fivept has Tsys=100 every time.

Note : my point about debug levels as in this example it would be nice to see if it's reading the right rxg file ! or which rxg file it is actually using. As there are others for different bands, these still have TCAL -100.

I have 4 rxg files, ie :

l.rxg : For L-Band, fixed is set to 1130 cl.rxg : For 4.8 GHz, fixed it set to 4000 c.rxg : For C-Band 6.7GHz, fixed it set to 5843 x.rxg : For X-Band 8-9GHz, fixed is set to 8000

For this I would expect it to be using "c.rxg" as I am using bands around 6.7 Ghz.

[wehimwich]

On May 18, 2021, at 1:34 AM, Eskil Varenius <@varenius> wrote:

On 2021-05-17 23:07, Stuart Weston wrote:

For this I would expect it to be using "c.rxg" as I am using bands around 6.7 Ghz.

I would expect the system to match the LO you set against the LO in the files. Given that you have "fixed" in your files, the LO should/must match exactly (otherwise you need/should use a range in your RXG file). Perhaps (just idea) if the system is not finding any matching RXG file (i.e. LO is set to some value not matching the file) it would assume Tcal -100?

[wehimwich]

On May 18, 2021, at 5:41 AM, Stuart Weston <@stuartdweston> wrote:

The LO in the RXG file is the one being used :

"fixed 5843"

If I change things in this file like TCAL then I get a corresponding change on the ONOFF output.

Add the LO 5843 to the DBBC BBC Freq settings and they match with the one's listed in the TCAL table.

[wehimwich]

@stuartdweston, to make long story short, this looks like it is a bug in 9.13.2 that was fixed in 10.0.0. If so, I forgot it was fixed and will add it to the Changes from FS9 document. I'm sorry about that. It is extremely unlikely it will be fixed in 9.13.x (never more). Some of the longer story follows.

I looked at the oriona.log (using the older Tcal value of 90), you sent, sent by email. It looks like that although fivpt is producing incorrect Tsys values, that is not interfering with its primary function. It does look like it is still able to find peaks. So this isn't quite just a cosmetic issue, but of course it isn't right.

Also from the log it looks like fivpt is getting a positive Tcal and it is not trying to control the cal. That should mean it is using continuous cal. The timing and values in the onoff records in the log agree with that. Can you confirm you are on continuous cal?

Also from the log, it looks like this system is still on 9.13.2, so this isn't a question of something not making it to 10.0.0 properly. Looking at the 9.13.2 code, it does look like there is a bug in continuous cal support that would cause this. That particular bug seems to be fixed in 10.0.0.

I think you have said that you remember this working correctly before. Could it be that was for non-continuous cal and/or with 10.0.0?

[stuartdweston]

Thanks Ed - keep the chatter here. I have my "test" FS with 10 on it. I'll try that shortly

Could it be that was for non-continuous cal and/or with 10.0.0?

It would have been an earlier version prior to 9.13.2, can't remember which one as it was 2-3 yrs ago.

[stuartdweston]

I have tried FS 10.0.0 with the same flux.ctl etc copied from our FS 9.13.02 system, controlling the 30m antenna

fivept no long produces a tsys of 100 :

2021.141.02:19:30.13;Log Opened: Mark IV Field System Version 10.0.0
2021.141.02:19:30.13;release,10.0.0-dirty
2021.141.02:19:30.13:uname,Linux,4.19.0-10-amd64,#1 SMP Debian 4.19.132-1 (2020-07-24),x86_64
2021.141.02:19:30.13;fortran,f95
2021.141.02:19:30.13;location,WARKWRTH,185.34,-36.43,90.0
2021.141.02:19:30.13;horizon1,0.,10.,360.
2021.141.02:19:30.13;antenna,30.5,22.0,21.6,-179.0,354.0,6.0,90.0,azel
2021.141.02:19:30.13;equip,dbbc_ddc/fila10g,flexbuff,none,none,500.10,3,a/d,101,60,20,none,40,1,in,8bit,50001:localhost,3,return,v106,v12,1,1,1,1,38000,38000,38000,38000,32,vsi2,v121,16,8
2021.141.02:19:30.13;time,0.000,1.000,computer
2021.141.02:19:30.13;flagr,200
2021.141.02:19:30.13;rdbe,20.0,12.0,28.0,normalized
2021.141.02:19:30.13;fsserver,disabled
2021.141.02:19:46.51;fivept
2021.141.02:19:46.51#fivpt#source taurusa    053432.0 +220058 2000.0 2021.141.02:19:46
2021.141.02:19:46.51#fivpt#site WARKWRTH 185.3370 -36.4332   0.00 xxxx   0  1.00  0.00
2021.141.02:19:46.51#fivpt#fivept azel   1   9 0.30 10 2u     9.9  0.1052     457.8
2021.141.02:19:46.51#fivpt#origin   0.0000   0.0000   0.0000   0.0000   0.0000   0.0000
2021.141.02:19:50.21&calofffp/sy=rte_go fivpt &
2021.141.02:19:50.21&calofffp/sy=rte_go fivpt &
2021.141.02:20:01.60#fivpt#tsys 354.204  31.368 405.891 19.3569
2021.141.02:20:16.61#fivpt#lat   1   4206.  -0.1611  -72.600  7.615
2021.141.02:20:30.34#fivpt#lat   2   4213.  -0.1208  -73.010  7.352
2021.141.02:20:44.06#fivpt#lat   3   4219.  -0.0805  -53.915  8.919
2021.141.02:20:57.81#fivpt#lat   4   4226.  -0.0403    2.163  6.671
2021.141.02:21:11.57#fivpt#lat   5   4233.   0.0000   71.343  7.261
2021.141.02:21:25.32#fivpt#lat   6   4240.   0.0403   57.766  9.163
2021.141.02:21:39.09#fivpt#lat   7   4247.   0.0805   -4.871  8.340
2021.141.02:21:52.86#fivpt#lat   8   4254.   0.1208  -43.902  7.159
2021.141.02:22:06.63#fivpt#lat   9   4261.   0.1611  -56.057  7.146
2021.141.02:22:06.63#fivpt#latfit   0.01530  0.1109 141.852 -65.433  0.3623   6
2021.141.02:22:06.63#fivpt#laterr   0.00112  0.0032  2.9443  1.8715  0.0660   3.0690
2021.141.02:22:20.60#fivpt#lon   1   4268.  -0.1886  -31.428 15.781
2021.141.02:22:34.40#fivpt#lon   2   4275.  -0.1415  -56.175 14.855
2021.141.02:22:48.21#fivpt#lon   3   4282.  -0.0943  -36.496 13.499
2021.141.02:23:02.05#fivpt#lon   4   4288.  -0.0472   31.147 10.421
2021.141.02:23:15.94#fivpt#lon   5   4295.   0.0000   71.801  7.190
2021.141.02:23:29.84#fivpt#lon   6   4302.   0.0472   22.721  8.616
2021.141.02:23:43.71#fivpt#lon   7   4309.   0.0943  -37.449 10.116
2021.141.02:23:57.56#fivpt#lon   8   4316.   0.1415  -56.116  7.362
2021.141.02:24:11.38#fivpt#lon   9   4323.   0.1886  -60.746  6.695
2021.141.02:24:11.38#fivpt#lonfit  -0.00004  0.0947 126.261 -52.043 -0.3064   9
2021.141.02:24:11.38#fivpt#lonerr   0.00395  0.0090  9.2661  4.8571  0.1857   9.3570
2021.141.02:24:11.38#fivpt#perform   12.790  1471.7    1.044    2.571
2021.141.02:24:13.98#fivpt#offset   354.3584   31.3249  -0.00004   0.01530  1  1
2021.141.02:24:13.98#fivpt#xoffset  354.3584   31.3249  -0.00003   0.01530  0.00338  0.00112 1 1 2u   taurusa

405.891 seem's a tad high ... But at least it's not 100.

[stuartdweston]

onoff I can almost believe :

2021.141.02:27:29.21#onoff#    source       Az   El  De   I P   Center   Comp   Tsys  SEFD  Tcal(j) Tcal(r)
2021.141.02:27:29.21#onoff#VAL taurusa    353.2 31.2 1u   1 l   6524.00 0.9887 302.1 1242.2  40.584  0.41
2021.141.02:27:29.21#onoff#VAL taurusa    353.2 31.2 2u   1 l   6534.00 0.9984 284.3 1008.9  35.033  0.35
2021.141.02:27:29.21#onoff#VAL taurusa    353.2 31.2 3u   1 l   6544.00 1.0021 246.7  969.3  38.775  0.39
2021.141.02:27:29.21#onoff#VAL taurusa    353.2 31.2 4u   1 l   6554.00 0.9926 242.9  971.1  39.463  0.40
2021.141.02:27:29.21#onoff#VAL taurusa    353.2 31.2 5u   1 l   6664.00 -0.060 214.4  991.0  45.615  0.46
2021.141.02:27:29.21#onoff#VAL taurusa    353.2 31.2 6u   1 l   6674.00 -3.313 215.8  990.6  45.301  0.46
2021.141.02:27:29.21#onoff#VAL taurusa    353.2 31.2 7u   1 l   6684.00 0.9978 218.6  993.1  44.833  0.45
2021.141.02:27:29.21#onoff#VAL taurusa    353.2 31.2 8u   1 l   6694.00 -0.120 223.9  996.1  43.909  0.44
2021.141.02:27:29.21#onoff#    source       Az   El  De   I P   Center   Comp   Tsys  SEFD  Tcal(j) Tcal(r)

[varenius]

Yay, progress, great stuff!

[wehimwich]

@stuartdweston, if you can please attach a short log with a fivpt and onoff run with echo=on, I will check at least the fivpt tsys calculation.

[stuartdweston]

fivpt / onoff on taurusa in C-Band github_125.log

[stuartdweston]

Here is one of the new sources I have added to flux.ctl. In C-Band 6.7 GHz it should be 8.478 Jy, the 30m should see this and has in the past.

*               freq MHz  ---- flux 10** ----   "     FS
* source  type  min  max  log   log(f)  2log(f) size  model
   0723m008 p  6400  6800 1.198 -0.04557 -0.3223   1  gauss 100 1s

For some reason today It just can't pick it out. Weather is pritty good, we have a cold southerly no rain and clear blue sky's. 0723m008.log

[wehimwich]

@stuartdweston, thank you for attaching github_125.log. I looked at the fivpt and onoff data briefly. The raw and reduced data from the fivpt Tsys measurement are:

2021.145.01:49:14.83#fivpt#fivept azel   1   9 0.30 10 2u     9.9  0.1052     457.8

2021.145.01:49:20.70#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,6603,4735,6775,4959\n>
2021.145.01:49:21.73#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,7033,4986,6857,4760\n>
2021.145.01:49:22.76#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,6931,4923,6754,4699\n>
2021.145.01:49:23.80#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,6807,4921,6640,4701\n>
2021.145.01:49:24.83#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,6768,4921,6588,4695\n>
2021.145.01:49:25.86#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,6596,4895,6422,4673\n>
2021.145.01:49:26.90#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,6272,4872,6102,4649\n>
2021.145.01:49:27.93#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,5931,4888,5759,4661\n>
2021.145.01:49:28.96#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,5744,4876,5574,4651\n>
2021.145.01:49:31.00#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,37,43,5504,4841,5327,4618\n>
2021.145.01:49:31.00#fivpt#tsys 358.231  31.534 445.680 39.2805

With a spreadsheet using the Tcal value of 9.9 (and only the CalOff value in the numerator of the calculation), I get 446.945 for Tsys, which agrees to the printing precision of Tcal. So I think the Tsys calculation is okay.

(Note that only using the CalOff value in the numerator is a small bug in the code that lowers the reported Tsys level by about one-half the Tcal value, about 5K in this case. This only affects the background level that Tant for the source is measured against. The background level is removed as part of the fitting process for the peak anyway, so the pointing results are unaffected. (Perhaps this bug has some benefit since it makes it less likely that Tant will be negative, which is always a little unpleasant.) It does affect the Tsys derived values of the #fivpt#perform record, but those are mostly a curiosity and typically not used for any significant work. Still it should probably be fixed (I created a new issue, #131), but that is not the issue here. OTOH, there are some things to say about the data set, which I will do after a brief look at the onoff data.)

Looking at the onoff data. The raw and reduced data for the first onsource point for 2U is:

2021.145.01:53:29.59#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,12276,12047,11983,11685\n>
2021.145.01:53:30.62#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,12377,12021,12083,11670\n>
2021.145.01:53:31.65#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,12440,11974,12142,11612\n>
2021.145.01:53:32.67#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,12356,11902,12064,11546\n>
2021.145.01:53:33.70#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,12328,11863,12031,11505\n>
2021.145.01:53:34.72#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,12062,11827,11769,11467\n>
2021.145.01:53:35.75#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,12240,11815,11939,11455\n>
2021.145.01:53:36.77#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,12208,11887,11916,11530\n>
2021.145.01:53:37.80#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,11883,11898,11608,11543\n>
2021.145.01:53:38.83#dbbcn#[dbbc02\0]<dbbc02/683.000000,a,16,1,man,48,54,11784,11898,11490,11547\n>

2021.145.01:53:38.83#onoff#ONSO     6.7   0.00000   0.00000 1u    11930.0    270.6 2u    11902.5    214.8

2021.145.01:53:38.83#onoff#ONSC     6.7   0.00000   0.00000 1u    12202.3    271.8 2u    12195.4    218.4

A spreadsheet calculation of the average and standard deviation (of the sample) agree to the printing precision in the log. I think that validates the basic data collection. The full "high-level" data collected and reduced values for 2U are:

2021.145.01:53:27.48#onoff#    De      Center  TCal    Flux    DPFU     Gain    Product   LO    T   FWHM
2021.145.01:53:27.48#onoff#APR 2u     6534.00 9.872  457.80  0.100000 1.00000  0.100000  5843.00 c 0.10522
2021.145.01:53:38.83#onoff#ONSO     6.7   0.00000   0.00000 1u    11930.0    270.6 2u    11902.5    214.8
2021.145.01:53:38.83#onoff#ONSC     6.7   0.00000   0.00000 1u    12202.3    271.8 2u    12195.4    218.4
2021.145.01:53:53.00#onoff#OFFC    20.8   0.47276   0.00000 1u    10320.4     92.1 2u     9434.9    613.6
2021.145.01:53:53.00#onoff#OFFS    20.8   0.47276   0.00000 1u    10060.0     93.0 2u     9144.4    615.0
2021.145.01:53:53.00#onoff#ZERO    20.8   0.47276   0.00000 1u        0.0      0.0 2u        0.0      0.0
2021.145.01:54:07.10#onoff#ONSO    34.9   0.00000   0.00000 1u     9928.8    141.2 2u    10779.1    156.8
2021.145.01:54:07.10#onoff#ONSC    34.9   0.00000   0.00000 1u    10196.4    141.7 2u    11065.2    159.6
2021.145.01:54:21.19#onoff#OFFC    49.0  -0.47276  -0.00000 1u     7233.6     99.8 2u     6119.2     70.7
2021.145.01:54:21.19#onoff#OFFS    49.0  -0.47276  -0.00000 1u     6971.3     98.2 2u     5832.6     71.6
2021.145.01:54:35.40#onoff#ONSO    63.2   0.00000   0.00000 1u    14420.3    448.8 2u    10690.7    116.8
2021.145.01:54:35.40#onoff#ONSC    63.2   0.00000   0.00000 1u    14679.7    449.0 2u    10979.5    118.1
2021.145.01:54:35.40#onoff#SIG                       2u                 11.981 3056.  931.3 401.131  4.06
2021.145.01:54:35.40#onoff#    source       Az   El  De   I P   Center   Comp   Tsys  SEFD  Tcal(j) Tcal(r)
2021.145.01:54:35.40#onoff#VAL taurusa    357.9 31.5 2u   1 l   6534.00 1.0025 261.1  961.1  36.335  0.37

A spreadsheet calculation gives the same Tsys and SEFD to the printing precision in the log.

Whew! It is nice once and awhile to reverify that these calculations are correct (and we found a small bug).

Now about the data in the log. It has some interesting features. At the highest level, the fivpt data is similar to what was pasted in a comment above. The Tant values while scanning the source are often strongly negative. This can be an indication the Tsys measurement (which is just used to set the scale for Tant) had strong RFI and/or was picking up another object (the Moon has occasionally been discovered). In those cases, the effect on the background level is mostly a constant (and maybe some scale error). We can see that the level is varying from point-to-point in the scan. It seems more likely that there is time and/or direction dependent interference.

Looking at the raw data for the fivpt 2u Tsys: its scatter is about 8% and more than three times the size of the CalOn vs CalOff difference. Not a good sign. OTOH, the adjacent 2L channel (not being used by fivpt) has a scatter of about 2% and a Tsys value, 265K, about half of that of 2U's and is more like the onoff result. BTW, is this receiver cooled?

Also, the raw data are not noise-like on this time scale. In particular, the 2U data is varying systematically. Does this receiver have gain variations on the scale of seconds? Or maybe this is varying pickup of some RFI. It seems a little large to just be atmospheric fluctuations, especially since it is so different in adjacent channels, but that may rule out gain variations too.

I may be wrong, but I thought we typically used a a target level of 16000 for DBBC TP measurements. We aren't near that, particularly for the fivpt Tsys data. It looks like the IF target levels are set, but perhaps a bit high (actual 42892 vs target 32000), but I don't see a read-out or setting of the BBC target levels. I suppose it is possible the DBBC is not operating in a good regime, but I thought we did this mostly for resolution. The latter is probably not an issue here.

Lastly, looking at the high level onoff data, there is something odd. The off source data on the two sides differ significantly, around, 9000 vs 6000 counts (the raw data appear to agree), a third to half of Tsys depending on how you look at it. The on source values seem to have 10+% variation as well. The SIG values for Tsys and SEFD are quite high as a result. While SIG values (estimates of the standard deviation) aren't always useful as an indication of a problem (or the lack there of), they seem to confirm that there is something going on here.

Anyway, I don't think this is a fivpt, (or onoff) issue. As a result I will close this issue (please feel free to continue to comment). I have attached a spreadsheet, github_125.xlsx, which includes the calculations mentioned above and some simple plots of the raw fivpt Tsys data.