search

DUNE / NTupleProd

Tools for producing ntuple files
1 stars 0 forks source link

document fcl 2 ways #2

Open hschellman opened 2 years ago

hschellman commented 2 years ago

I do fhicl-dump --annotate $TOPFCL > $JOBDIR/job-$TOPFCL.txt to store the full thick

Also useful to add a sam parameter that stores the fcl file path

whereis -S $FHICL_FILE_PATH -f filename

does it.

hschellman commented 2 years ago

Maybe 2? filename and the path, then you can search for them separately.

brettviren commented 2 years ago

To access documentation on what config an art plugin expects:

$ art --print-description WCLS  -c cfg/cosmic_volts.fcl

====================================================================================================

    tool_type: WCLS (or "larwirecell/Tools/WCLS")

        provider: user
        type    : class
        source  : / [ external source ] /larwirecell/Tools/WCLS_tool.cc
        library : /cvmfs/larsoft.opensciencegrid.org/products/larwirecell/v09_04_00/slf7.x86_64.e20.prof/lib/liblarwirecell_Tools_WCLS_tool.so

    Allowed configuration
    ---------------------

        ## Any parameters prefaced with '#' are optional.

        <tool_label>: {

           tool_type: WCLS

           ## List of one or more WCT configuration files.
           ## They are located w.r.t. the WCT load path.

           configs: [
              <string>,
              ...
           ]

           ## List of one or more WCT application objects to execute.

           apps: [
              <string>,
              ...
           ]

           ## List of WCT component plugin libraries to load.
           ## They are located w.r.t. the OS library load path

           plugins: [
              <string>,
              ...
           ]

           ## Optional list of file system paths to add to the WCT configuration file load path.
           ## This augments the WIRECELL_PATH environment variable.

         # paths: [
         #    <string>,
         #    ...
         # ]

           ## Optional table giving external variables to inject into WCT configuration.

         # params: << delegated >>

           ## Optional table giving external Jsonnet code to inject into WCT configuration.

         # structs: << delegated >>

           ## List of WCT components which act as WCT sources.
           ## They are called before WCT executes on each Art Event object

         # inputers: [
         #    <string>,
         #    ...
         # ]

           ## List of WCT components which act as WCT sinks.
           ## They are called after WCT executes on each Art Event object.

         # outputers: [
         #    <string>,
         #    ...
         # ]

           ## List of WCT log sinks.
           ## Eg the strings 'stdout', 'stderr' or a file name.
           ## An optional log level may be appended with ':<level>'.

         # logsinks: [
         #    <string>,
         #    ...
         # ]

           ## List of minimum WCT logger levels.
           ## Specify as '<logger>:<level>' or as just '<level>' for default.

         # loglevels: [
         #    <string>,
         #    ...
         # ]
        }

====================================================================================================
hschellman commented 2 years ago

Thanks, knew it was someplace in there.

We're trying to write a lar wrapper for batch jobs that logs stuff like this so one knows what one did, and for how long and with what resources.

On 10/7/21 11:00 AM, Brett Viren wrote:

[This email originated from outside of OSU. Use caution with links and attachments.]

To access documentation on what config an art plugin expects:

$ art --print-description WCLS -c cfg/cosmic_volts.fcl

====================================================================================================

tool_type: WCLS (or "larwirecell/Tools/WCLS")

    provider: user
    type    : class
    source  : / [ external source ] /larwirecell/Tools/WCLS_tool.cc
    library : /cvmfs/larsoft.opensciencegrid.org/products/larwirecell/v09_04_00/slf7.x86_64.e20.prof/lib/liblarwirecell_Tools_WCLS_tool.so

Allowed configuration
---------------------

    ## Any parameters prefaced with '#' are optional.

    <tool_label>: {

       tool_type: WCLS

       ## List of one or more WCT configuration files.
       ## They are located w.r.t. the WCT load path.

       configs: [
          <string>,
          ...
       ]

       ## List of one or more WCT application objects to execute.

       apps: [
          <string>,
          ...
       ]

       ## List of WCT component plugin libraries to load.
       ## They are located w.r.t. the OS library load path

       plugins: [
          <string>,
          ...
       ]

       ## Optional list of file system paths to add to the WCT configuration file load path.
       ## This augments the WIRECELL_PATH environment variable.

     # paths: [
     #    <string>,
     #    ...
     # ]

       ## Optional table giving external variables to inject into WCT configuration.

     # params: << delegated >>

       ## Optional table giving external Jsonnet code to inject into WCT configuration.

     # structs: << delegated >>

       ## List of WCT components which act as WCT sources.
       ## They are called before WCT executes on each Art Event object

     # inputers: [
     #    <string>,
     #    ...
     # ]

       ## List of WCT components which act as WCT sinks.
       ## They are called after WCT executes on each Art Event object.

     # outputers: [
     #    <string>,
     #    ...
     # ]

       ## List of WCT log sinks.
       ## Eg the strings 'stdout', 'stderr' or a file name.
       ## An optional log level may be appended with ':<level>'.

     # logsinks: [
     #    <string>,
     #    ...
     # ]

       ## List of minimum WCT logger levels.
       ## Specify as '<logger>:<level>' or as just '<level>' for default.

     # loglevels: [
     #    <string>,
     #    ...
     # ]
    }

====================================================================================================

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDUNE%2FNTupleProd%2Fissues%2F2%23issuecomment-938027149&data=04%7C01%7Cheidi.schellman%40oregonstate.edu%7C68e2c197c6904ca8dce908d989bc51b6%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637692264201876943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=OqZO%2Flp7ZB%2FEQkNQUeUBxiJTwf5QRq95DC4%2BNBtHRQk%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIA37DOW3V5RAY42ZYMUQRLUFXN2NANCNFSM5FR2M56Q&data=04%7C01%7Cheidi.schellman%40oregonstate.edu%7C68e2c197c6904ca8dce908d989bc51b6%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637692264201886898%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GWcSwf3%2FXre1ROexkmdy%2FWkeAk8t0iCe97JG9dlAeh0%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cheidi.schellman%40oregonstate.edu%7C68e2c197c6904ca8dce908d989bc51b6%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637692264201886898%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Vj6td6SycQf6pkQAY0pLcNY3bkAbOB3%2Fu01p5dhSL2g%3D&reserved=0 or Androidhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cheidi.schellman%40oregonstate.edu%7C68e2c197c6904ca8dce908d989bc51b6%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C637692264201896859%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2Ftt%2FKB5peMsIxcCogsMowj6TZ1bIv5%2FJQ2aVH0J0k58%3D&reserved=0.

hschellman commented 2 years ago

Brett, I'm still confused. Turns out your response didn't cure my particular problem . We need to document where a fcl file came from. So we need to know the exact directory it resided in. This tells me why it does but not where it is. This aspect of fcl files terrifies me for reproducibility. If the source of your config depends on the phase of the moon, or how you set up your path and you can't trace it, you can't reproduce your results. I can use the Whereis method but it's failing for Jake so we're hoping there is some better way of asking art where exactly it got that fcl from.

brettviren commented 2 years ago

Yes, the 

art --print-description <thing> [...]

call is about documenting what FHiCL configuration is possible for a given art plugin. It doesn't speak to what FHiCL configuration is already existing. I was trying to add more information about the topic "document fcl 2 ways" but maybe i guessed wrong about the intended meaning and this contribution is off-topic.