search

nasa-nccs-hpda / srlite

surface reflectance
0 stars 1 forks source link

Add "filename" output column [CSV] #19

Closed gtamkin closed 1 month ago

gtamkin commented 6 months ago

[PM] In the CSVs, add an output column called "filename", where the entry would be the srlite tif basename, like "WV02_20200120_M1BS_10300455DVA3400-sr-2m.tif" that when a whole boatload of CSVs are concatenated into 1 data frame, we have that identifying info right in the table,

gtamkin commented 3 months ago

image Paul added requirements on 3/19/24.

gtamkin commented 1 month ago
image
gtamkin commented 1 month ago

FYI. Here are examples of how these files look.

Tamkin, Glenn S. (GSFC-606.3)[InuTeq, LLC]: Examples of SR-Lite Statistics CSV and... sent on March 19, 2024 12:23 PM

From: "Montesano, Paul M. (GSFC-606.3)[ADNET SYSTEMS INC]" [paul.m.montesano@nasa.gov](mailto:paul.m.montesano@nasa.gov) Date: Tuesday, March 19, 2024 at 11:08 AM To: "Tamkin, Glenn S. (GSFC-606.3)[InuTeq, LLC]" [glenn.s.tamkin@nasa.gov](mailto:glenn.s.tamkin@nasa.gov), "Carroll, Mark L. (GSFC-6063)" [mark.carroll@nasa.gov](mailto:mark.carroll@nasa.gov) Subject: RE: SR-Lite output clarification #2: optional or mandatory CSV files?

I think those should be made by default. They don’t take up space and it should be part of any best practices usage of srlite results.

Can you drop me an example of how these files looks? Just drop it into Teams – I need to refresh my memory. -p

From: Tamkin, Glenn S. (GSFC-606.3)[InuTeq, LLC] [glenn.s.tamkin@nasa.gov](mailto:glenn.s.tamkin@nasa.gov) Sent: Tuesday, March 19, 2024 11:00 AM To: Carroll, Mark L. (GSFC-6063) [mark.carroll@nasa.gov](mailto:mark.carroll@nasa.gov); Montesano, Paul M. (GSFC-606.3)[ADNET SYSTEMS INC] [paul.m.montesano@nasa.gov](mailto:paul.m.montesano@nasa.gov) Cc: Tamkin, Glenn S. (GSFC-606.3)[InuTeq, LLC] [glenn.s.tamkin@nasa.gov](mailto:glenn.s.tamkin@nasa.gov) Subject: SR-Lite output clarification #2: optional or mandatory CSV files?

Mark, Paul,

While we are on the topic of user parameters… should the CSV Statistics file and new CSV ERROR report be created by default? Or should the user have to specify a flag to initiation each?

Right now, the ERROR report is output by default, but the Statistics file is only created if the user specifies a flag.

Thanks, Glenn

gtamkin commented 1 month ago
image
gtamkin commented 1 month ago

,index,band_name,model,intercept,slope,r2_score,explained_variance,mbe,mae,mape,medae,mse,rmse,mean_ref_sr,mean_toa_sr,mae_norm,rmse_norm,catid,input_toa,input_ref,input_cloudmask 0,0,Blue,rma,-0.08842676609070743,1.1286322824174266,0.947089228701615,0.9470892287299886,8.270534936973667e-07,0.025097187796316933,0.07143681388715636,0.017699999999999994,0.001275549014162217,0.035714829051280886,0.35648282456928354,0.35648199751578974,0.07040223558214995,0.1001866754574584,1030010049148A00,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.tif,/panfs/ccds02/nobackup/people/iluser/projects/s rlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-ccdc.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.cloudmask.v1.2.tif 1,0,Green,rma,-0.03953198994885515,1.1057025183741809,0.9750076395078418,0.9750076395108489,-2.7864189110492053e-07,0.018799192354399218,0.04823702634088724,0.013800000000000034,0.0006452897803858287,0.02540255460353995,0.4192710431465186,0.41927132178840953,0.04483780280487829,0.060587429107673355,1030010049148A00,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.tif,/panfs/ccds02/nobackup/people/iluser/proje cts/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-ccdc.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.cloudmask.v1.2.tif 2,0,Red,rma,-0.02676459822584576,1.0555923409533754,0.9608934927968998,0.9608934927981623,-1.9657422964534125e-07,0.024198045070672455,0.05319604689183257,0.017199999999999993,0.0011970386297195832,0.03459824604975783,0.48157315195271144,0.4815733485269409,0.05024791139737085,0.07184421704047829,1030010049148A00,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.tif,/panfs/ccds02/nobackup/people/iluser/projects /srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-ccdc.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.cloudmask.v1.2.tif 3,0,NIR,rma,0.010801263674221406,1.0284316276299934,0.9693691853519976,0.9693691853520343,-2.994360620805937e-08,0.01991554572222315,0.039629623883821644,0.014400000000000024,0.0007458193074154786,0.027309692554393184,0.5345345726659236,0.5345346026095299,0.03725773175511713,0.0510906009656018,1030010049148A00,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.tif,/panfs/ccds02/nobackup/people/iluser/projects/s rlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-ccdc.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.cloudmask.v1.2.tif 4,0,Coastal,rma,-0.08842676609070743,1.1286322824174266,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,1030010049148A00,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-ccdc.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.cloudmask.v1.2.tif 5,0,Yellow,rma,-0.0328035745108292,1.0792944548734162,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,1030010049148A00,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-ccdc.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.cloudmask.v1.2.tif 6,0,RedEdge,rma,-0.012527136565720302,1.0452984306038138,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,1030010049148A00,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-ccdc.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.cloudmask.v1.2.tif 7,0,NIR2,rma,0.010801263674221406,1.0284316276299934,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,1030010049148A00,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-ccdc.tif,/panfs/ccds02/nobackup/people/iluser/projects/srlite/test/input/baseline/WV02_20150911_M1BS_1030010049148A00-toa.cloudmask.v1.2.tif

gtamkin commented 1 month ago

Add several items to initial filename change as per PM.