Open kamilliano opened 5 years ago
Hi @kamilliano thanks for providing a detailed bug report. My guess is that json-ld plugin is not registered properly on Lambda as it should, see https://github.com/RDFLib/rdflib-jsonld#using-the-plug-in-jsonld-serializerparser-with-rdflib , likely using this bit of code https://github.com/RDFLib/rdflib-jsonld/blob/070d45cad067276e72df5d8f362aee65c158df40/setup.py#L106-L113 using this machinery https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins, so there must be a way to register it manually
I think it's possible to register plugins directly via https://github.com/RDFLib/rdflib/blob/1503dae6049f0d9b14d9d7f884d8de4cb38b39a3/rdflib/plugin.py#L88 , like this https://github.com/RDFLib/rdflib/blob/1503dae6049f0d9b14d9d7f884d8de4cb38b39a3/rdflib/plugin.py#L164-L166 - do you think you can try this @kamilliano ?
@lopuhin Thanks for quick response. I will have a stab at this.
@lopuhin I did try to register 'json-ld' plugin directly and that works. Thanks for that I owe you a beer. I just didn't fully understand the internals. I think my problem is more from my side and to do with the project structure and how imports are managed in some places (I haven't written all myself so I am not taking blame for all this time ;) ). I tried to inspect globals and locals between local environment and that of lambda, everything seems to be there, but can't fully understand how pipes fit together at runtime...
@kamilliano great that it worked, thanks for digging into it! If possible, can you share the bit of code to register the plugins, so that if someone else has the same problem, they can find your solution?
I think my problem is more from my side and to do with the project structure and how imports are managed in some places
I'm not sure, but I don't think it's the issue with your project code - plugin should be registered automatically without any extra imports or anything else. I suspect that this could be AWS Lambda's fault here (in the way they handle python package installation), or the fault of the tool which packages everything for Lambda - not quite sure how it works. But to check this properly one would probably need to create a very simple and small project that would demonstrate that.
@lopuhin the following is the registration that I used in the module calling the main function above:
from rdflib.plugin import register
from rdflib.serializer import Serializer
register(
'json-ld', Serializer,
'rdflib_jsonld.serializer', 'JsonLDSerializer')
Should we cover this in the documentation and consider the issue fixed once that is done? There does not seem to be much else we can do.
I’ve just tried to use extruct in Amazon Lambda myself.
Because extruct depends on lxml, you need a Docker image. From then on, it should all work as in a regular Linux machine, so I don’t think there’s anything we need to cover in the documentation, unless there is enough interest to cover this in a FAQ section.
Hi, wanted to ask if anyone out there has used extruct on AWS lambda? I tested running
extruct
function which seems to fail to work for rdfa. Other default metadata types are fine.A simple test case:
The part of pipenv graph for extruct when I build the artifact.zip file:
When I run this locally in the same pipenv env (Ubuntu 17.10, Docker, 17.12.0-ce, pipenv==v2018.11.26), I don't experience any issues. On lambda invocation I log the following stack trace:
I have been scratching my head over this but can't figure this one out. What should I try? Thanks in advance