Support for READ-only filesystems

[mdeng10]

updating to address https://github.com/mpenning/ciscoconfparse/issues/267

[mdeng10]

15

sorry i don't understand?

[mpenning]

@mdeng10

I'm not sure I fully grok all this PR hopes to accomplish, but at this point we're suggesting that we modify CiscoConfParse for a problem I cannot even reproduce yet. I do not have an AWS account and do not believe it should be required. Is there a way to reproduce with a simple vmware or other vm image I can download somewhere?

Alternatively if you want to give me a public key via email to login somewhere that's also an option. You can email the public key to me and make a note in this ticket when you do as sometimes my mail hits my spam folder.

[mdeng10]

@mpenning

Is there a way to reproduce with a simple vmware or other vm image I can download somewhere?

sure - logically since this bug only happens on read-only file systems you should be able to just:

1. just spin up any (linux) VM
2. pip install ciscoconfparse
3. write a python script that imports it: below should work

from ciscoconfparse import CiscoConfParse

def main()
  print("hello world")

# if main section unnecssary, fails on the import

4. and this is the main part - set the entire system + all directories to read only for the current user (so don't run the python script as root, obviously) this should work but i haven't tested it sudo chmod -R 555 / otherwise you might need to do sudo chmod -R 444 / but unlikely IMO
5. attempt to run the python script - python main.py or whatever you named it

[mdeng10]

yes it fails on the import of ciscoconfparse so the main() is unnecessary, it won't print, the error occurs on the import of ciscoconfparse - this is what the error trace stack looks like

"  File \"/var/task/core/main.py\", line 7, in <module>\n    from ciscoconfparse import CiscoConfParse\n",
    "  File \"/opt/python/ciscoconfparse/__init__.py\", line 30, in <module>\n    from ciscoconfparse.ciscoconfparse import *\n",
    "  File \"/opt/python/ciscoconfparse/ciscoconfparse.py\", line 240, in <module>\n    initialize_ciscoconfparse()\n",
    "  File \"/opt/python/loguru/_logger.py\", line 1226, in catch_wrapper\n    return function(*args, **kwargs)\n",
    "  File \"/opt/python/ciscoconfparse/ciscoconfparse.py\", line 228, in initialize_ciscoconfparse\n    configure_loguru()\n",
    "  File \"/opt/python/loguru/_logger.py\", line 1226, in catch_wrapper\n    return function(*args, **kwargs)\n",
    "  File \"/opt/python/ciscoconfparse/ccp_util.py\", line 313, in configure_loguru\n    ccp_logger_control(\n",
    "  File \"/opt/python/loguru/_logger.py\", line 1226, in catch_wrapper\n    return function(*args, **kwargs)\n",
    "  File \"/opt/python/ciscoconfparse/ccp_util.py\", line 248, in ccp_logger_control\n    logger.add(\n",
    "  File \"/opt/python/loguru/_logger.py\", line 945, in add\n    handler = Handler(\n",
    "  File \"/opt/python/loguru/_handler.py\", line 86, in __init__\n    self._queue = multiprocessing.SimpleQueue()\n",
    "  File \"/var/lang/lib/python3.8/multiprocessing/context.py\", line 113, in SimpleQueue\n    return SimpleQueue(ctx=self.get_context())\n",
    "  File \"/var/lang/lib/python3.8/multiprocessing/queues.py\", line 336, in __init__\n    self._rlock = ctx.Lock()\n",
    "  File \"/var/lang/lib/python3.8/multiprocessing/context.py\", line 68, in Lock\n    return Lock(ctx=self.get_context())\n",
    "  File \"/var/lang/lib/python3.8/multiprocessing/synchronize.py\", line 162, in __init__\n    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)\n",
    "  File \"/var/lang/lib/python3.8/multiprocessing/synchronize.py\", line 57, in __init__\n    sl = self._semlock = _multiprocessing.SemLock(\n"

are you able to share more details regarding your replication steps to confirm they're the same?

[mpenning]

I see… have I mentioned that I am deaf? :-) Well, this PR could be useful for your use-case… I am considering it.

[mdeng10]

basically it's just a dependency on the multithreading which AWS lambda does not support, i'd be happy with any solution to be honest, at the moment we're just shipping this a modified version in AWS lambda but it's inconvenient as we have to pip install -r requirements.txt and then remove ciscoconfparse, add the custom version

[mpenning]

I see… have I mentioned that I am deaf? :-) Well, this PR could be useful for your use-case… I am considering it.

I considered it. Sadly I cannot agree to supporting read-only filesystems.

[mdeng10]

I see… have I mentioned that I am deaf? :-) Well, this PR could be useful for your use-case… I am considering it.

I considered it. Sadly I cannot agree to supporting read-only filesystems.

That's fair enough - is there any workaround so that we can use ciscoconfparase on read-only filesystems? is this a relevant fix https://github.com/mpenning/ciscoconfparse/issues/215#issuecomment-986870153

[mpenning]

you mentioned AWS lambda, but I'm curious are you trying to run ciscoconfparse on an embedded-system?

[mdeng10]

no, just AWS lambda + possibly read-only containers in the future at the moment