post/download protocol taking relpath as absolute path

[benlapETS]

When posting with srpost (watch) there is no way of making the relpath not starting with / which sr\<protocol> will take as absolute path and then never find the file, for example with sftp, posting a path and with a watch and subscribing to it with those simple config (and many variants) will never work:

• sr_watch:

  
  # this is to post mocked file created by ualf0-to-ec99 subscriber
  #

post_broker amqp://tfeed@${FLOWBROKER}/ post_exchange xpublic

path ${LIGHTNING_HOME}

events create,modify

post_base_url sftp://ben@localhost/ post_base_dir ${LIGHTNING_HOME}

* sr_subscribe:

#

Sample subscribe config, mocking an old feed from a newer one

broker amqp://tfeed@${FLOWBROKER} exchange xpublic report_exchange xpublic

subtopic *.VAISALA.LIGHTNING.#

directory ${LIGHTNING_HOME}/lightning/mocked mirror True

accept ./VAISALA/LIGHTNING/ascii/.

[petersilva]

Please experiment with document_root and post_document_root they may solve this issue

[benlapETS]

I first experimented with that and and I was still having troubles. The documentation is really unclear what to do with that. This is an issue to make this clearer.

[benlapETS]

Using document_root:

• sr_watch

  
  # this is to post mocked file created by ualf0-to-ec99 subscriber
  #

post_broker amqp://tfeed@${FLOWBROKER}/ post_exchange xpublic

path ${LIGHTNING_HOME}

events create,modify

post_base_url file: post_base_dir ${LIGHTNING_HOME} post_document_root ${LIGHTNING_HOME}

* sr_subscribe

#

Sample subscribe config, mocking an old feed from a newer one

broker amqp://tfeed@${FLOWBROKER} exchange xpublic report_exchange xpublic

document_root ${LIGHTNING_HOME}

subtopic *.VAISALA.LIGHTNING.#

directory ${LIGHTNING_TEST}/mocked mirror True

accept ./VAISALA/LIGHTNING/ascii/.

* Got that (the file is there)

2019-07-08 14:16:35,102 [WARNING] /20190708/VAISALA/LIGHTNING/ascii/18/CLDN~20190708181800:Vaisala_cldnget:CMC:Lightning:ASCII moved or removed since announced 2019-07-08 14:16:35,103 [INFO] file_log downloaded to: /local/ben/lightning_test/mocked/20190708/VAISALA/LIGHTNING/ascii/18/CLDN~20190708181800:Vaisala_cldnget:CMC:Lightning:ASCII

[benlapETS]

As a workaround for my case I changed only lines 783+ in sr_post (and removed document_root) to:

        # relpath
        self.post_relpath = path
        if self.post_base_dir :
           self.post_relpath = path.replace(self.post_base_dir, '')
           if self.post_relpath.startswith('/'):
               self.post_relpath = self.post_relpath[1:]

And it worked for me, but as our code is not modular enough it will surely breaks something elsewhere... so I am not proposing to fix it like that, just as a mean of understanding what is going on. In fact, it worked with sftp: and not with file:.