search

EranOfek / AstroPack

Astronomy & Astrophysics Software Pacakge
Other
19 stars 4 forks source link

Scheduler demon doesn't load correctly target files #513

Closed EastEriq closed 2 weeks ago

EastEriq commented 3 weeks ago

Even after my minor changes of b0aa83b9a, I have not been able to load correctly the target file made today by Ruslan. To test load it with telescope.Scheduler.demon('TargetList','/home/ocs/ObservationSchedule.csv'). The file is on last0. My first change of line 834 is because it appears that you cannot just assign a table to an empty object; however even after that workaround (which might be incorrect), the demon stumbles onto

Operator '-' is not supported for operands of type 'table'.

Error in telescope.Scheduler/get.HA (line 262)
            Val       = Obj.LST - Obj.RA;

Error in telescope.Scheduler/weight (line 1834)
            HA_Day  = Obj.HA./360;  % [day]

Error in telescope.Scheduler/selectTarget (line 1733)
            W   = Obj.weight(JD);

Error in telescope.Scheduler/serviceTargetRequests (line 947)
                [TargetInd, Priority, Tbl, Struct] = S.selectTarget(JD,...

Error in telescope.Scheduler.demon (line 1041)
                S.serviceTargetRequests('ToO_File',Args.ToO_File,...

the moment is has to dispatch a target. The reason is that at that point Obj.RA is a table, whereas with the default survey tiling (i.e. calling just telescope.Scheduler.demon) it is an array of doubles, which can be then subtracted from Obj.LST. I'm not familiar with all the mechanics of catalogs, so please help.

@RuslanKonno : next time, please dry-run the lists at day time (S.send("Unit.observeAskingTargets('NoHardware',true,'SimulationJD',celestial.time.julday+0.5)"))

EastEriq commented 3 weeks ago

Shorter way to reproduce the problem: before https://github.com/EranOfek/AstroPack/commit/b0aa83b9ad697da290803ff798d269b24a4e1946

>> S=telescope.Scheduler;
>> S.loadTable('ObservationSchedule.csv');
Expected one output from a curly brace or dot indexing expression, but there were 0
results.

Error in telescope.Scheduler/loadTable (line 838)
                    ExistFieldName = Obj.List.Catalog.(Obj.ColFieldName);

After https://github.com/EranOfek/AstroPack/commit/b0aa83b9ad697da290803ff798d269b24a4e1946

>> S=telescope.Scheduler;
>> S.loadTable('ObservationSchedule.csv','replace');
>> S.selectTarget()
Operator '-' is not supported for operands of type 'table'.

Error in telescope.Scheduler/get.HA (line 262)
            Val       = Obj.LST - Obj.RA;

Error in telescope.Scheduler/weight (line 1834)
            HA_Day  = Obj.HA./360;  % [day]

Error in telescope.Scheduler/selectTarget (line 1733)
            W   = Obj.weight(JD);
EranOfek commented 3 weeks ago

Fixed using the following modification loadTable now updates Ntarget and Ncol Not sure what exactly happened, but it is working now.

EastEriq commented 3 weeks ago

There are still bugs:

  1. this one is still present 
    
>> S=telescope.Scheduler;
>> S.loadTable('ObservationSchedule.csv');
Expected one output from a curly brace or dot indexing expression, but there were 0 results.

Error in telescope.Scheduler/loadTable (line 842) ExistFieldName = Obj.List.Catalog.(Obj.ColFieldName);

2.

S.loadTable('ObservationSchedule.csv','replace'); class(S.List.Catalog.FieldName(1))

ans =

'double'

it should be a string. It might be because in @RuslanKonno's csv the content is written without quotes, but the mechanism should be made a bit more robust. Leaving it as a number affects the log message, the image filename and what not. All can be taken care downstream but it s better to fix it at the time of ingesting the table.
EranOfek commented 3 weeks ago

Problem 1 now fixed. Problem 2 - in the new version of loadTables, if the FieldNames is numeric then it will be converted to strings.

EastEriq commented 3 weeks ago

To move on I committed a dirty fix for the second in 444f29521. Note this further bug:

>> S=telescope.Scheduler;
>> S.loadTable('ObservationSchedule.csv','replace');
>> S.loadTable('ObservationSchedule.csv');
Error using telescope.Scheduler/loadTable (line 857)
Index exceeds the number of array elements (1535).
EranOfek commented 3 weeks ago

bug fixed. [dev1 dc117bdcf]

EastEriq commented 3 weeks ago

You can then remove the num2str I inserted at my previous commit?

Il 4 nov 2024 14:03, Eran Ofek @.***> ha scritto: Caution: External Sender. Do not click on links or open attachments unless you recognize the sender.

bug fixed. [dev1 dc117bdhttps://github.com/EranOfek/AstroPack/commit/dc117bdcfe6210131acbbce3b6de51f3fc0eff10]

— Reply to this email directly, view it on GitHubhttps://github.com/EranOfek/AstroPack/issues/513#issuecomment-2454535623, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD2SPYZQEAR6TBJONUXYH33Z65O77AVCNFSM6AAAAABRC4TC2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJUGUZTKNRSGM. You are receiving this because you modified the open/close state.Message ID: @.***>

EranOfek commented 3 weeks ago

can't recreate error

On Mon, Nov 4, 2024 at 1:06 PM EastEriq @.***> wrote:

To move on I committed a dirty fix for the second in 444f295 https://github.com/EranOfek/AstroPack/commit/444f29521abac53314d9c94e7ce8c9de596a785d . Note this further bug:

S=telescope.Scheduler; S.loadTable('ObservationSchedule.csv','replace'); S.loadTable('ObservationSchedule.csv'); Error using telescope.Scheduler/loadTable (line 857) Index exceeds the number of array elements (1535).

— Reply to this email directly, view it on GitHub https://github.com/EranOfek/AstroPack/issues/513#issuecomment-2454422982, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUQ4OQAVYH4BXUAL6JVN3Z65IMXAVCNFSM6AAAAABRC4TC2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJUGQZDEOJYGI . You are receiving this because you modified the open/close state.Message ID: @.***>

EranOfek commented 3 weeks ago

removed. In general the best way to store characters in a table is via string array (not cell array and not chars).

On Mon, Nov 4, 2024 at 2:26 PM EastEriq @.***> wrote:

You can then remove the num2str I inserted at my previous commit?

Il 4 nov 2024 14:03, Eran Ofek @.> ha scritto:
Caution: External Sender. Do not click on links or open attachments unless you recognize the sender.


bug fixed.
[dev1 dc117bd< https://github.com/EranOfek/AstroPack/commit/dc117bdcfe6210131acbbce3b6de51f3fc0eff10&gt;]

—
Reply to this email directly, view it on GitHub< https://github.com/EranOfek/AstroPack/issues/513#issuecomment-2454535623&gt;, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AD2SPYZQEAR6TBJONUXYH33Z65O77AVCNFSM6AAAAABRC4TC2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJUGUZTKNRSGM&gt;.
You are receiving this because you modified the open/close state.Message ID: @.>

— Reply to this email directly, view it on GitHub https://github.com/EranOfek/AstroPack/issues/513#issuecomment-2454585060, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUQ4NAZVFNFEMJ5JZJDCTZ65RXFAVCNFSM6AAAAABRC4TC2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJUGU4DKMBWGA . You are receiving this because you modified the open/close state.Message ID: @.***>

EranOfek commented 3 weeks ago

but this introduced a new error: S=telescope.Scheduler;

S.loadTable('ObservationSchedule.csv','replace'); S.loadTable('ObservationSchedule.csv'); Error using setdiff First argument must be text.

Error in telescope.Scheduler/loadTable (line 854) [~,Ic] = setdiff(NewFieldName, ExistFieldName);

checking

On Mon, Nov 4, 2024 at 2:26 PM EastEriq @.***> wrote:

You can then remove the num2str I inserted at my previous commit?

Il 4 nov 2024 14:03, Eran Ofek @.> ha scritto:
Caution: External Sender. Do not click on links or open attachments unless you recognize the sender.


bug fixed.
[dev1 dc117bd< https://github.com/EranOfek/AstroPack/commit/dc117bdcfe6210131acbbce3b6de51f3fc0eff10&gt;]

—
Reply to this email directly, view it on GitHub< https://github.com/EranOfek/AstroPack/issues/513#issuecomment-2454535623&gt;, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AD2SPYZQEAR6TBJONUXYH33Z65O77AVCNFSM6AAAAABRC4TC2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJUGUZTKNRSGM&gt;.
You are receiving this because you modified the open/close state.Message ID: @.>

— Reply to this email directly, view it on GitHub https://github.com/EranOfek/AstroPack/issues/513#issuecomment-2454585060, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUQ4NAZVFNFEMJ5JZJDCTZ65RXFAVCNFSM6AAAAABRC4TC2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJUGU4DKMBWGA . You are receiving this because you modified the open/close state.Message ID: @.***>

EastEriq commented 3 weeks ago

Can you guarantee that one way or the other of loading the table works by this evening?

EranOfek commented 3 weeks ago

Fixed using new function: tools.string.convert2strings [dev1 355ea5c80]

simonegarrappa commented 2 weeks ago

Now with alphanumeric target names it returns a NaN that is registered in the filename/header. e.g. target for mount 5 '4Cp2750' becomes 'NaN' as a string.

>> telescope.Scheduler.demon('TargetList','/home/ocs/ObservationSchedule.csv')
15:41:50.159 [INF] selecting target for mount 1, requested at JD=2460620.154047, now 2460620.154053
15:41:51.664 [INF] Mount=  1  Target =                 1392  RA=334.285714  Dec= 33.000000 Priority=  1.20  Nexp= 20 ExpTime= 20.0
15:41:51.692 [INF] selecting target for mount 5, requested at JD=2460620.154050, now 2460620.154070
15:41:52.713 [INF] Mount=  5  Target =                  NaN  RA=350.499426  Dec= 27.546234 Priority=  3.10  Nexp=400 ExpTime= 20.0
15:41:53.305 [INF] selecting target for mount 4, requested at JD=2460620.154057, now 2460620.154089
15:41:54.254 [INF] Mount=  4  Target =                  NaN  RA= 30.950000  Dec= 31.340000 Priority=  3.10  Nexp= 20 ExpTime= 20.0
15:41:54.269 [INF] selecting target for mount 6, requested at JD=2460620.154059, now 2460620.154100
15:41:56.370 [INF] Mount=  6  Target =                 1234  RA=336.428571  Dec= 21.000000 Priority=  1.20  Nexp= 20 ExpTime= 20.0
15:41:56.400 [INF] selecting target for mount 10, requested at JD=2460620.154064, now 2460620.154125
15:41:57.406 [INF] Mount= 10  Target =                 1314  RA=331.111111  Dec= 27.000000 Priority=  1.20  Nexp= 20 ExpTime= 20.0
15:41:57.424 [INF] selecting target for mount 2, requested at JD=2460620.154066, now 2460620.154137
EranOfek commented 2 weeks ago

Is the file name also contains NaN? Or is it ok?

On Tue, 5 Nov 2024 at 17:50 Simone Garrappa @.***> wrote:

Now with alphanumeric target names it returns a NaN that is registered in the filename/header. e.g. target for mount 5 '4Cp2750' becomes 'NaN' as a string.

telescope.Scheduler.demon('TargetList','/home/ocs/ObservationSchedule.csv') 15:41:50.159 [INF] selecting target for mount 1, requested at JD=2460620.154047, now 2460620.154053 15:41:51.664 [INF] Mount= 1 Target = 1392 RA=334.285714 Dec= 33.000000 Priority= 1.20 Nexp= 20 ExpTime= 20.0 15:41:51.692 [INF] selecting target for mount 5, requested at JD=2460620.154050, now 2460620.154070 15:41:52.713 [INF] Mount= 5 Target = NaN RA=350.499426 Dec= 27.546234 Priority= 3.10 Nexp=400 ExpTime= 20.0 15:41:53.305 [INF] selecting target for mount 4, requested at JD=2460620.154057, now 2460620.154089 15:41:54.254 [INF] Mount= 4 Target = NaN RA= 30.950000 Dec= 31.340000 Priority= 3.10 Nexp= 20 ExpTime= 20.0 15:41:54.269 [INF] selecting target for mount 6, requested at JD=2460620.154059, now 2460620.154100 15:41:56.370 [INF] Mount= 6 Target = 1234 RA=336.428571 Dec= 21.000000 Priority= 1.20 Nexp= 20 ExpTime= 20.0 15:41:56.400 [INF] selecting target for mount 10, requested at JD=2460620.154064, now 2460620.154125 15:41:57.406 [INF] Mount= 10 Target = 1314 RA=331.111111 Dec= 27.000000 Priority= 1.20 Nexp= 20 ExpTime= 20.0 15:41:57.424 [INF] selecting target for mount 2, requested at JD=2460620.154066, now 2460620.154137

— Reply to this email directly, view it on GitHub https://github.com/EranOfek/AstroPack/issues/513#issuecomment-2457543256, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUQ4JRZZOMOT2GT5OZWHTZ7DSM7AVCNFSM6AAAAABRC4TC2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJXGU2DGMRVGY . You are receiving this because you modified the open/close state.Message ID: @.***>

RuslanKonno commented 2 weeks ago

Also, all targets with a dot extension are stripped of the dot extension. e.g. "1234.ABC" in the .csv file becomes becomes "1234" when passed to mounts.

simonegarrappa commented 2 weeks ago

When it is misconverted as 'NaN' there is no field name in the image file.

e.g. LAST.01.05.01_20241105.155807.508_clear__048_001_001_sci_raw_Image_1.fits

EastEriq commented 2 weeks ago

Is the file name also contains NaN? Or is it ok?

Ah , I though that the filename is built from Camera.Object which is set to FieldName, so I would have expected NaN there too.

image

EranOfek commented 2 weeks ago

Fixed by modifying read2table - adding:

Opts = detectImportOptions(Data);
Ifn = find(strcmp(Opts.VariableNames, 'FieldName'));
Opts.VariableTypes{Ifn} = 'string';
Data = readtable(Data, Opts);

[dev1 21f273476]