nanotate-tool / nanotate

Nanotate is a free and open source tool. The code behind the tool is available on Github. End-users can install it by creating a bookmark. The documentation about how to install and use the Nanotate is available at http://doi.org/10.5281/zenodo.5101941 A running instance of the tool can be found https://nanotate.bitsfetch.com
https://nanotate.bitsfetch.com
Apache License 2.0
3 stars 0 forks source link

Use python nanopub library to sign and publish RDF #1

Closed raar1 closed 3 years ago

raar1 commented 3 years ago

Hi @oxgiraldo and @miguel-ruano ,

First of all, let me just say this is a very nice tool!

Based on some discussion we had on Tuesday, I wanted to suggest using the nanopub python library for publishing the assertions you are generating - we have been developing it here: https://github.com/fair-workflows/nanopub

It is on pypi, so you can add it as a dependency in requirements.txt (it's quite lightweight). Current version is 0.2.2, but we are actively developing it every day with a few devs, so if you have any issues or need any features then please just let us know.

So signing and publishing an assertion (such as you are already creating) could be done as follows:

from nanopub import Nanopub, NanopubClient

# Create the client, that allows searching, fetching and publishing nanopubs
client = NanopubClient()

# Build whatever RDF assertion you want, as an rdflib graph
# my_assertion = ...

# Make a Nanopub object with this assertion
nanopub = Nanopub.from_assertion(assertion_rdf=my_assertion)

# Publish the Nanopub object. The URI at which it is published is returned.
publication_info = client.publish(nanopub)

Note that the from_assertion() function has several optional parameters: introduces_concept, derived_from, attributed_to, nanopub_author etc so you can customize more than just the assertion. We're also adding 'user profiles' to the library, so that you can always use the same identity while publishing, which might be useful to you later on. There are also nanopub search methods in the library, in case you are interested in adding search functionality to your tool (such as searching for existing nanotate publications).

I hope this can at least help simplify the nanopub generation and publishing part of your backend. I am more than happy to help, solve bugs etc if you run into trouble too. Let me know what you think!

miguel-ruano commented 3 years ago

Hi @raar1 @oxgiraldo

testing nanopub on our bakckend this worked fine !!

in this process I have a question,

when the nanopublication with reference is published: https://protocolexchange.researchsquare.com/article/pex-1069/v1 the result that it returns to me at the end of the publication is:

{
   'nanopub_uri': 'https://protocolexchange.researchsquare.com/article/pex-1069/v1#RAl6W1vijjCZIHiKNUk1F_bq1uLVCpYhAb6wsvU8NEHVk'
}

and prints on the screen

1 nanopub published at http://server.nanopubs.lod.labs.vu.nl/

So with all this my question is how do I get the final access url to the nanopublicacion at http://server.nanopubs.lod.labs.vu.nl/? according my tests the final url would be something like that http://server.nanopubs.lod.labs.vu.nl/RAl6W1vijjCZIHiKNUk1F_bq1uLVCpYhAb6wsvU8NEHVk

a suggestion would be that in the response of publish function

client = NanopubClient()
publication_info = client.publish(nanopub)

will return more information about the publication, for example

{
  'nanopub_uri': 'https://protocolexchange.researchsquare.com/article/pex-1069/v1#RAl6W1vijjCZIHiKNUk1F_bq1uLVCpYhAb6wsvU8NEHVk',
  'artifact_code': 'RAl6W1vijjCZIHiKNUk1F_bq1uLVCpYhAb6wsvU8NEHVk',
  'published_at': 'http://server.nanopubs.lod.labs.vu.nl',
  'canonical_url': 'http://server.nanopubs.lod.labs.vu.nl/RAl6W1vijjCZIHiKNUk1F_bq1uLVCpYhAb6wsvU8NEHVk'
}

I remain attentive to your comments.

raar1 commented 3 years ago

Hi @miguel-ruano , thanks a lot for your reply. I must apologise as I've been on holiday for a week, and I am still catching up with all my messages. I will try to reply to your points soon!

raar1 commented 3 years ago

Hi @miguel-ruano, do you have an example of the nanopub command you are using to create the nanopub? I assume you are setting the uri parameter in from_assertion() (or are you modifying the rdf directly?).

In general, that uri kwarg should not be set, as the trustification of the URI requires something of the form http://purl.org/nanopub/temp/mynanopub and normally you do not want or need to change that. This URI is essentially just a 'dummy' URI which is overwritten during 'trustification' of the URI by the nanopub publishing tool. So I recommend simply to leave it unset, and check what published URI is returned.

The really weird URI you are currently getting back is because the java tool expects a URI like the purl.org one above, otherwise it tacks on a bunch of weird stuff to the end. I think the reason you wanted to set the URI in the first place is to show that it is derived from a given protocol/paper/etc? We discussed this a little yesterday evening with Olga and Tobias, and it was suggested that this should in fact go in the pubinfo or provenance subgraphs. But we are still working this out, I think.

raar1 commented 3 years ago

Ah, so I see there is a DBNanopub class that constructs the rdf you want to use for the whole nanopub. You set the assertion subgraph, and you add triples to the provenance subgraph, such as:

                (
                    self.np.assertion,
                    GraphNanopub.PROV.wasDerivedFrom,
                    rdflib.URIRef("https://hypothes.is/a/" + component.id),
                )

If you like, you can simplify this part of your code using the nanopub library:

nanopub = Nanopub.from_assertion(
        assertion_rdf=self.assertion,
        derived_from=rdflib.URIRef("https://hypothes.is/a/" + component.id)
)

This would use the assertion rdf (that is processed by __computeAssertion() I think), and set the wasDerivedFrom to what you want. Then if you do client.publish(nanopub) you will get a (non-weird) published URI back. Note that you can also add other things if you need them (like nanopub_author, attributed_to etc) using this from_assertion() function.

I think this way you can greatly simplify the DBNanopub stuff as well. I hope what I've written above makes sense, but I have not had enough coffee this morning so... :)

miguel-ruano commented 3 years ago

hi @raar1

yeah, i was modifying the rdf directly. Then I made the changes according to your comments to use the nanopub more and it already gives me a url of the type

http://purl.org/np/RAu7dNMHoTY0Ns8e8yzlqlNxgcHGRSofAoa-pUi1_ZVak

I have made updates to make it more dependent of nanopub and leave all the rdf to it. only that in the use of the Nanopub.from_assertion the derived_from parameter cannot be sent as a list, and our nanopublications is derived from one or many annotations made in hypothesis.

so we have to edit only this part of the rdf directly. see in code

self.nanopub = Nanopub.from_assertion(
    assertion_rdf=self._computeAssertion(), nanopub_author=author
)
self._computeProvenance(derived_from)

our own derived from. see in code

def _computeProvenance(self, derived_from: list):
        """
        computes wasDeriveFrom for many
        """
        assertion_uri = self.np.assertion
        for derived_from_item in derived_from:
            self.provenance.add(
                (
                    assertion_uri,
                    GraphNanopub.PROV.wasDerivedFrom,
                    rdflib.URIRef(derived_from_item),
                )
            )

but now it is more dependent on nanopub

miguel-ruano commented 3 years ago

@raar1 i have at one question , i can delete or disable published nanopubs ?

raar1 commented 3 years ago

Hi @miguel-ruano, ah I see - ok I have quickly added a high priority issue to nanopub to allow derived_from to take a list of URIs instead of a single one. You can see here if you want to add anything to the discussion later: https://github.com/fair-workflows/nanopub/issues/42

raar1 commented 3 years ago

Unfortunately the nanopub concept does not seem to include deletion of published nanopubs. You can 'retract' them, but I think this is essentially publishing another nanopublication that says 'nanopub X is retracted' so it's probably not what you wanted.

Are you worried about 'polluting' the nanopub servers with too many test publications? We recently set up a test server (http://test-grlc.nanopubs.lod.labs.vu.nl/api/local/local/) that clears all its contents every so often. To use it, simply set use_test_server=True when you create the nanopub client, i.e. client = NanopubClient(use_test_server=True) and all published nanopubs end up there.

Would that help a bit?

raar1 commented 3 years ago

@miguel-ruano There is now a new release of nanopub: version 0.2.8, that allows you to pass a list to derived_from in from_assertion(). This will generate a prov:wasDerivedFrom triple for each URI provided. Would you be able to update your requirements.txt to use the new version and test this out?

One annoying thing for you, I'm afraid - we changed the class name from Nanopub to Publication to avoid confusion with the name of the library itself. So your code will now look like e.g.

from nanopub import Publication, NanopubClient

client = NanopubClient()

nanopub = Publication.from_assertion(assertion_rdf=my_assertion, derived_from=['uri1', 'uri2', 'uri3'])

Sorry about that, we're working towards our first main release of the library, after which we do not want to be changing class names anymore. So it means we need to fix any confusions now. If you also notice anything that is unclear, please let us know.

Have a nice weekend!

miguel-ruano commented 3 years ago

hi @raar1

i made the update of nanopub in nanotate and make changes according of the new version and works perfect!!!, thanks for the update,

testing in test mode, i see something when publishing the nanopublication the return url is like http://purl.org/np/RAdG9VXVOBq-7Ju5H085CgynfrVlwpajLb02B-W0ss8Zc, when i go to url this returns 404(not found), because the nanopub not is publish in http://server.nanopubs.lod.labs.vu.nl/ if not in http://test-server.nanopubs.lod.labs.vu.nl/

seeing the nanopub code exactly in the NanopubClient publish method

the process is the following first sign the rdf and then publish it

in the process of sign the rdf, the code seems not know about the test enviroment and always url base is like http://purl.org/np/RAdG9VXVOBq-7Ju5H085CgynfrVlwpajLb02B-W0ss8Zc

def sign(self, unsigned_file: Union[str, Path]) -> str:
    unsigned_file = str(unsigned_file)
        self._run_command(f'{NANOPUB_SCRIPT} sign ' + unsigned_file)
        return self._get_signed_file(unsigned_file)

see in code

so when extract the nanopub uri, after publishing in test server

 @staticmethod
 def extract_nanopub_url(signed: Union[str, Path]):
      # Extract nanopub URL
      # (this is pretty horrible, switch to python version as soon as it is ready)
      extracturl = rdflib.Graph()
      extracturl.parse(str(signed), format="trig")
      return dict(extracturl.namespaces())['this'].__str__()

see in code

always return the same url for test or production mode, i guess that bug is in the sign method.

I hope this helps you to solve it in the future.

regards,

raar1 commented 3 years ago

Hi @miguel-ruano, thanks! Yes I'm afraid you're quite right about the test server URI problem. Tobias set up the server very recently, so there are still some issues to iron out, partly because we don't yet know what the test server will look like long term. I will add this as an issue to nanopub for now. I hope it will be a quick fix.

Please don't hesitate to let me know of any other bugs or features that would be useful to have. (Or even add them directly as issues on our repo if you like - that's good too).

raar1 commented 3 years ago

@miguel-ruano We have made changes that should solve the test server issue you mention. Please use nanopub v0.2.9 now.

In the new version, as long as your client is set to use the test server (use_test_server=True) then the fetch() method will also check the test server for the requested nanopub. Note that the URI will still be the same as that of a normally published nanopub, so if you follow the URI directly it will not resolve unless the nanopub is available on the usual servers. But if you use client.fetch() it will load the nanopub correctly. (We decided this would be the most desirable behaviour for the testing mode.)

Let me know if this helps

miguel-ruano commented 3 years ago

hi @raar1 thanks for the updates, this was a great update!, i make the changes in our code for implements the new specifications and works fine,

the fetch for test mode is ok the retract is a great function too!, for updated or remove nanopublications

thanks

miguel-ruano commented 3 years ago

i was speaking with @oxgiraldo about creating a multi-step nanopub of a whole protocol, this is a new feature you suggest.

do i need use another library aside of nanopub?, or does this one already do it?

regards

raar1 commented 3 years ago

Hi Miguel, great, thanks! I'm glad the changes are helping.

Yes, indeed if you use the fairworkflows library (version 0.1.7) as a dependency then you can do the following:

from fairworkflows import FairWorkflow, FairStep

# Create a workflow
workflow = FairWorkflow(description='This is a test workflow.')

# Load some steps from nanopublications
preheat_oven = FairStep.from_nanopub(uri='http://purl.org/np/RACLlhNijmCk4AX_2PuoBPHKfY1T6jieGaUPVFv-fWCAg#step')
melt_butter = FairStep.from_nanopub(uri='http://purl.org/np/RANBLu3UN2ngnjY5Hzrn7S5GpqFdz8_BBy92bDlt991X4#step')
arrange_chicken = FairStep.from_nanopub(uri='http://purl.org/np/RA5D8NzM2OXPZAWNlADQ8hZdVu1k0HnmVmgl20apjhU8M#step')

# Specify ordering of steps
workflow.first_step = preheat_oven
workflow.add(melt_butter, follows=preheat_oven)
workflow.add(arrange_chicken, follows=melt_butter)

# Validates?
workflow.validate()

# Publish a new nanopublication of the workflow that links all these steps together
workflow.publish_as_nanopub()

Actually, you can also technically use the library to make nanopublications of steps from scratch (using the plex ontology) as follows:

from fairworkflows import FairStep
import rdflib

# Make a new 'empty' step
step = FairStep()

# Specify various characteristics needed to describe it
step.label = 'Slicing an union 🧅'
step.description = 'Slice the union in 0.5 cm thick slices'
step.is_manual_task = True

# Add other statements, about the step itself
step.set_attribute(predicate=rdflib.URIRef('http://example.org/needsEquipment'),
                   value=rdflib.URIRef('http://example.org/Knife'))

# Add any other, general triples
step.add_triple(rdflib.URIRef('http://example.org/Union'),
                rdflib.URIRef('http://example.org/Has'),
                rdflib.URIRef('http://example.org/Layers'))

# Set the URIs of the inputs and outputs to this step
step.inputs = ['http://example.org/IntactUnion', 'http://example.org/Knife']
step.outputs = ['http://example.org/SlicedUnion']

# Print the RDF description of the step
print(step)

# Publish the step as a nanopublication for others to find
step.publish_as_nanopub()

The following should also work if you want to use the test server:

step.publish_as_nanopub(use_test_server=True)

Basically all this library does is help build the rdf for steps and workflows, and then uses the nanopub library to publish them. But don't worry too much about all the features, I think we can maybe do something simple that lets us combine several of the step nanopubs you are already creating into a workflow nanopub too. That way you would have the whole protocol as its own publication.

raar1 commented 3 years ago

Sorry if that was too much detail - I will help you with any problems using this

miguel-ruano commented 3 years ago

great!

i will check the library and i tell you if i have any problem

regards

miguel-ruano commented 3 years ago

Hi Robin

i can't use fairworksflows lib because needs 'nanopub' version 0.2.7 , so this make a conflict with our 'backend' that need 'nanopub' version 0.2.9

they have any updates scheduled for this?

raar1 commented 3 years ago

Hi Miguel, yes, you're right, sorry about that. There is a new release of fairworkflows, version 0.1.8. That one uses the new nanopub library. Would you mind testing now?

miguel-ruano commented 3 years ago

Hi Robin, dependencies are now correct. Thanks!!

miguel-ruano commented 3 years ago

Hi Robin

testing the nanopub test server, we have a problem on publishing nanopublications.

in some one nanopublications, when trying publish them, the server returns error 'HTTP Status 400 – Bad Request' with message ''Nanopub is not trusty:"

tracking the code of this process, i was able to retrieve the rdf of nanopublication and the html response from server, for a random nanopublication

rdf of nanopublication

@prefix sub: <http://purl.org/np/RAIVrp-y0hQE0vNwY0x1w3rXLdguf6Wd0D4gfgArhIUp4#> .
@prefix authority: <https://hypothes.is/a/> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
@prefix np: <http://www.nanopub.org/nschema#> .
@prefix pav: <http://purl.org/pav/> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix npx: <http://purl.org/nanopub/x/> .

sub:Head {
  this: np:hasAssertion sub:assertion;
    np:hasProvenance sub:provenance;
    np:hasPublicationInfo sub:pubInfo;
    a np:Nanopublication .
}

sub:assertion {
  sub:step dc:description "mycobacteria such as BCG (Bacillus Calmette–Guérin) or Mycobacterium tuberculosis (M. tuberculosis)," .
}

sub:provenance {
  sub:assertion pav:createdWith <https://nanotate.bitsfetch.com>;
    prov:generatedAtTime "2020-12-15T21:43:17.735319"^^xsd:dateTime;
    prov:wasAttributedTo <https://hypothes.is/a/acct:iannotate@hypothes.is>;
    prov:wasDerivedFrom authority:PNkXND9FEeux7WO6Lc0lAA .
}

sub:pubInfo {
  sub:sig npx:hasAlgorithm "RSA";
    npx:hasPublicKey "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCAYBVNwcjF5IK+z2KRM4MqV5gG0zVrjoqZtICaimdnS2jGy8YCweNDHUD6ZWfv5p4dWQal1NAMUm1GRXJGwGYIi3ug/zL5YLU72QeDa1ziLjWa2UHHhdGzjoZy+TB87Zhe8LI4fJUXhU+jMwxCG2O613mXmmOpNQBGFJL6+VOyWwIDAQAB";
    npx:hasSignature "V4OZ7pxMyoyHU0VwuCt3aq2/swxG0h+vqHlrDBU4UVBPm+50Fao/I55EET8talFHkAdOj8fXk9NJ4i0rJyqWEQQBxpUeRu8ob4dgGKFyyVxjPgtLmLxUsfqzRVZ63sfBMos6XjWLogk5HHxg9RBklkmbL6hOpTiFvPrFUqkqe0Y=";
    npx:hasSignatureTarget this: .

  this: prov:generatedAtTime "2020-12-15T21:43:17.735319"^^xsd:dateTime;
    prov:wasAttributedTo <https://orcid.org/0000-0002-7241-7089> .
}

html from server response

<!doctype html>
<html lang="en">

<head>
    <title>HTTP Status 400 – Bad Request</title>
    <style type="text/css">
        h1 {
            font-family: Tahoma, Arial, sans-serif;
            color: white;
            background-color: #525D76;
            font-size: 22px;
        }

        h2 {
            font-family: Tahoma, Arial, sans-serif;
            color: white;
            background-color: #525D76;
            font-size: 16px;
        }

        h3 {
            font-family: Tahoma, Arial, sans-serif;
            color: white;
            background-color: #525D76;
            font-size: 14px;
        }

        body {
            font-family: Tahoma, Arial, sans-serif;
            color: black;
            background-color: white;
        }

        b {
            font-family: Tahoma, Arial, sans-serif;
            color: white;
            background-color: #525D76;
        }

        p {
            font-family: Tahoma, Arial, sans-serif;
            background: white;
            color: black;
            font-size: 12px;
        }

        a {
            color: black;
        }

        a.name {
            color: black;
        }

        .line {
            height: 1px;
            background-color: #525D76;
            border: none;
        }
    </style>
</head>

<body>
    <h1>HTTP Status 400 – Bad Request</h1>
    <hr class="line" />
    <p><b>Type</b> Status Report</p>
    <p><b>Message</b> Nanopub is not trusty:
        http:&#47;&#47;purl.org&#47;np&#47;RAIFLGA99mIlH4SytloJFhf5BxcZrkZTFHDpYqj0P_ge0</p>
    <p><b>Description</b> The server cannot or will not process the request due to something that is perceived to be a
        client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).
    </p>
    <hr class="line" />
    <h3>Apache Tomcat/8.5.41</h3>
</body>

</html>

we unknow what cause this issue. is it due to any update in nanopub lib?

thanks

raar1 commented 3 years ago

Hi @miguel-ruano ! So sorry about the late reply, I didn't see that this issue was updated and I completely forgot. We will look into this problem now.

raar1 commented 3 years ago

Hi Miguel - I assume the definition of this appears above sub in the nanopub RDF you've posted? Like @prefix this: <http://purl.org/np/RAIVrp-y0hQE0vNwY0x1w3rXLdguf6Wd0D4gfgArhIUp4> .?

If I add that to the trig I indeed also get the following if I try to publish it from the command line:

>>> np publish test.trig
FAILED TO PUBLISH NANOPUBS
raar1 commented 3 years ago

What's odd is that otherwise, np likes it:

>>> np check test.trig
Summary: 1 trusty with signature;
raar1 commented 3 years ago

This is my test.trig:

@prefix this: <http://purl.org/np/RAIVrp-y0hQE0vNwY0x1w3rXLdguf6Wd0D4gfgArhIUp4> .
@prefix sub: <http://purl.org/np/RAIVrp-y0hQE0vNwY0x1w3rXLdguf6Wd0D4gfgArhIUp4#> .
@prefix authority: <https://hypothes.is/a/> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
@prefix np: <http://www.nanopub.org/nschema#> .
@prefix pav: <http://purl.org/pav/> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix npx: <http://purl.org/nanopub/x/> .

sub:Head {
  this: np:hasAssertion sub:assertion;
    np:hasProvenance sub:provenance;
    np:hasPublicationInfo sub:pubInfo;
    a np:Nanopublication .
}

sub:assertion {
  sub:step dc:description "mycobacteria such as BCG (Bacillus Calmette–Guérin) or Mycobacterium tuberculosis (M. tuberculosis)," .
}

sub:provenance {
  sub:assertion pav:createdWith <https://nanotate.bitsfetch.com>;
    prov:generatedAtTime "2020-12-15T21:43:17.735319"^^xsd:dateTime;
    prov:wasAttributedTo <https://hypothes.is/a/acct:iannotate@hypothes.is>;
    prov:wasDerivedFrom authority:PNkXND9FEeux7WO6Lc0lAA .
}

sub:pubInfo {
  sub:sig npx:hasAlgorithm "RSA";
    npx:hasPublicKey "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCAYBVNwcjF5IK+z2KRM4MqV5gG0zVrjoqZtICaimdnS2jGy8YCweNDHUD6ZWfv5p4dWQal1NAMUm1GRXJGwGYIi3ug/zL5YLU72QeDa1ziLjWa2UHHhdGzjoZy+TB87Zhe8LI4fJUXhU+jMwxCG2O613mXmmOpNQBGFJL6+VOyWwIDAQAB";
    npx:hasSignature "V4OZ7pxMyoyHU0VwuCt3aq2/swxG0h+vqHlrDBU4UVBPm+50Fao/I55EET8talFHkAdOj8fXk9NJ4i0rJyqWEQQBxpUeRu8ob4dgGKFyyVxjPgtLmLxUsfqzRVZ63sfBMos6XjWLogk5HHxg9RBklkmbL6hOpTiFvPrFUqkqe0Y=";
    npx:hasSignatureTarget this: .

  this: prov:generatedAtTime "2020-12-15T21:43:17.735319"^^xsd:dateTime;
    prov:wasAttributedTo <https://orcid.org/0000-0002-7241-7089> .
}
raar1 commented 3 years ago

I also occasionally get this:

>>> np publish test.trig
13:47:26,550  INFO RetryExec:97 - I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {}->http://nanopubs.restdesc.org:80: The target server failed to respond
13:47:26,555  INFO RetryExec:113 - Retrying request to {}->http://nanopubs.restdesc.org:80
13:47:26,620  INFO RetryExec:97 - I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {}->http://nanopubs.restdesc.org:80: The target server failed to respond
13:47:26,620  INFO RetryExec:113 - Retrying request to {}->http://nanopubs.restdesc.org:80
13:47:26,696  INFO RetryExec:97 - I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {}->http://nanopubs.restdesc.org:80: The target server failed to respond
13:47:26,696  INFO RetryExec:113 - Retrying request to {}->http://nanopubs.restdesc.org:80
13:47:31,588  INFO RetryExec:97 - I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {}->http://nanopubs.restdesc.org:80: The target server failed to respond
13:47:31,589  INFO RetryExec:113 - Retrying request to {}->http://nanopubs.restdesc.org:80
13:47:31,671  INFO RetryExec:97 - I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {}->http://nanopubs.restdesc.org:80: The target server failed to respond
13:47:31,671  INFO RetryExec:113 - Retrying request to {}->http://nanopubs.restdesc.org:80
13:47:31,743  INFO RetryExec:97 - I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {}->http://nanopubs.restdesc.org:80: The target server failed to respond
13:47:31,743  INFO RetryExec:113 - Retrying request to {}->http://nanopubs.restdesc.org:80
FAILED TO PUBLISH NANOPUBS
raar1 commented 3 years ago

@miguel-ruano I'm not sure exactly what the problem is yet, but could you try upgrading to use nanopub version 1.2.0? That way I'm sure I'm running the same thing.

Also, what sort of server is Nanotate actually running on? Is it a windows image by any chance?

raar1 commented 3 years ago

(Also upgrade fairworkflows from 0.1.8 to 0.1.9 just in case)

miguel-ruano commented 3 years ago

hi @raar1 ,

I understand, I am going to update the libraries and I will tell you the results, as for our server it is a linux-ubuntu 20.04

raar1 commented 3 years ago

as for our server it is a linux-ubuntu 20.04

Ok, that's great, we shouldn't have any issues with a linux image. Let's see what happens with the newer version.

miguel-ruano commented 3 years ago

hi @raar1

I have updated the nanopub and fairworksflow libs and the error seems continue.

I clarify that this does not happen for all nanopublications made in nanotate, only in some is that this inconvenience occurs.

i will make a deep tracking of the process, I'll tell you if I find something

Regards

miguel-ruano commented 3 years ago

Hi robin

It seems that I have found the root of the problem!

It is something related to the charmap that is used when signing the nanopublication to be published in nanopub

for example I was trying to publish this nanopublication

initial nanopublication

@prefix : <http://purl.org/nanopub/temp/mynanopub#> .
@prefix authority: <https://hypothes.is/a/> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
@prefix np: <http://www.nanopub.org/nschema#> .
@prefix pav: <http://purl.org/pav/> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

:pubInfo {
    : prov:generatedAtTime "2021-01-11T11:04:32.705428"^^xsd:dateTime ;
        prov:wasAttributedTo <https://orcid.org/0000-0002-3784-0922> .

    :assertion prov:wasDerivedFrom authority:E_06qlNvEeuWRR_bAb5r-A .
}

:Head {
    : a np:Nanopublication ;
        np:hasAssertion :assertion ;
        np:hasProvenance :provenance ;
        np:hasPublicationInfo :pubInfo .
}

:assertion {
    :step dc:description "Centrifuge the tubes (without any delay) for 30 min at 100 x g in 4 °C in a swing-out bucket. Fixed angle rotors also can be used but would require more caution when separating cells in interphase." .
}

:provenance {
    :assertion pav:createdWith <https://nanotate.bitsfetch.com> ;
        prov:generatedAtTime "2021-01-11T11:04:32.705428"^^xsd:dateTime ;
        prov:wasAttributedTo <https://hypothes.is/a/acct:miguel.ruano@hypothes.is> .
}

and made lots of changes to the rdf structure but nothing seemed to work then I wondered if the problem was suddenly in the content of the step

Centrifuge the tubes (without any delay) for 30 min at 100 x g in 4 °C in a swing-out bucket. Fixed angle rotors also can be used but would require more caution when separating cells in interphase.

the only strange character of all these was ° I removed it from the text and sent the nanopublication

Nanopublication whit changes

@prefix : <http://purl.org/nanopub/temp/mynanopub#> .
@prefix authority: <https://hypothes.is/a/> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
@prefix np: <http://www.nanopub.org/nschema#> .
@prefix pav: <http://purl.org/pav/> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

:pubInfo {
    : prov:generatedAtTime "2021-01-11T11:04:32.705428"^^xsd:dateTime ;
        prov:wasAttributedTo <https://orcid.org/0000-0002-3784-0922> .

    :assertion prov:wasDerivedFrom authority:E_06qlNvEeuWRR_bAb5r-A .
}

:Head {
    : a np:Nanopublication ;
        np:hasAssertion :assertion ;
        np:hasProvenance :provenance ;
        np:hasPublicationInfo :pubInfo .
}

:assertion {
    :step dc:description "Centrifuge the tubes (without any delay) for 30 min at 100 x g in 4 C in a swing-out bucket. Fixed angle rotors also can be used but would require more caution when separating cells in interphase." .
}

:provenance {
    :assertion pav:createdWith <https://nanotate.bitsfetch.com> ;
        prov:generatedAtTime "2021-01-11T11:04:32.705428"^^xsd:dateTime ;
        prov:wasAttributedTo <https://hypothes.is/a/acct:miguel.ruano@hypothes.is> .
}

and works!, the nanopublication was published ok. then investigate the cause of this, so i find this is due to the charmap that being used, in the nanotate server it currently use the ANSI_X3.4-1968 encoding, i could`t change this settings for permissions issue, so modify the java execution parameters and it would be something like this

exec java -Dsun.jnu.encoding=utf8 -Duser.language=en -Duser.country=US -Dfile.encoding=utf8 -jar $NANOPUBJAR "$@"

(this change was made in the file nanopub-java that is in the bin folder, tracking the java_wrapper.py and there this file is executed)

with this i setting the charmap in utf8 , publish again the initial nanopublication and this works ok!

So i have a question, what charmap is use in the nanopub server (in this case test-server) ? i don't know if it's utf8 the right charmap, but seems that it works fine for now

Regards

raar1 commented 3 years ago

Hi @miguel-ruano , that's great! Thanks for debugging this. It's kind of weird, but I think this is probably because we've been using different Java implementations, and the one we're using is already assuming UTF-8, or something along those lines. We need to look into this though to try to make the library more robust. Out of interest, which Java are you using?

@tkuhn Do you have any past experience of this UTF-8/Java issue?

miguel-ruano commented 3 years ago

java version i use is java version "1.8.0_211" Java(TM) SE Runtime Environment (build 1.8.0_211-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)

raar1 commented 3 years ago

Hmm, ok, on my side I have this:

>>> java --version
java 11.0.6 2020-01-14 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.6+8-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.6+8-LTS, mixed mode)

Not sure if that's the cause, but I don't get this UTF-8 issue with the above. Need to investigate this.

tkuhn commented 3 years ago

nanopub-java assumes and uses UTF-8. But strange that this is causing problems. I also can't find your nanopublication that you claimed was successfully published in the network. Was it really published?

miguel-ruano commented 3 years ago

the nanopublication was published in test server http://test-server.nanopubs.lod.labs.vu.nl/ nanopublication uri is : http://purl.org/np/RAkZ8r3l45lZz65-gpir4GjiABuzfUwSBSFZ6Yv95Zyu8# view in : http://test-server.nanopubs.lod.labs.vu.nl/RAkZ8r3l45lZz65-gpir4GjiABuzfUwSBSFZ6Yv95Zyu8#

tkuhn commented 3 years ago

Right, it's on the test server. That's good then.

So, to narrow that down, you can make the nanopub trusty (with np mktrusty or similar) but then not publish it? What happens if you try to check the nanopub before publishing (with np check)?

miguel-ruano commented 3 years ago

Hi @tkuhn

I made your suggestions and these are the results

first let me show the settings of my machine

global machine charmap settings

$ locale charmap
ANSI_X3.4-1968

java settings

$ java -XshowSettings -version
VM settings:
    Max. Heap Size (Estimated): 1.78G
    Ergonomics Machine Class: server
    Using VM: Java HotSpot(TM) 64-Bit Server VM

Property settings:
    awt.toolkit = sun.awt.X11.XToolkit
    file.encoding = ANSI_X3.4-1968
    file.encoding.pkg = sun.io
    file.separator = /
    java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
    java.awt.printerjob = sun.print.PSPrinterJob
    java.class.path = .
    java.class.version = 52.0
    java.endorsed.dirs = /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/endorsed
    java.ext.dirs = /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/ext
        /usr/java/packages/lib/ext
    java.home = /home/bitsfetc/virtualenv/java/jre1.8.0_211
    java.io.tmpdir = /tmp
    java.library.path = /usr/java/packages/lib/amd64
        /usr/lib64
        /lib64
        /lib
        /usr/lib
    java.runtime.name = Java(TM) SE Runtime Environment
    java.runtime.version = 1.8.0_211-b12
    java.specification.name = Java Platform API Specification
    java.specification.vendor = Oracle Corporation
    java.specification.version = 1.8
    java.vendor = Oracle Corporation
    java.vendor.url = http://java.oracle.com/
    java.vendor.url.bug = http://bugreport.sun.com/bugreport/
    java.version = 1.8.0_211
    java.vm.info = mixed mode
    java.vm.name = Java HotSpot(TM) 64-Bit Server VM
    java.vm.specification.name = Java Virtual Machine Specification
    java.vm.specification.vendor = Oracle Corporation
    java.vm.specification.version = 1.8
    java.vm.vendor = Oracle Corporation
    java.vm.version = 25.211-b12
    line.separator = \n
    os.arch = amd64
    os.name = Linux
    os.version = 3.10.0-962.3.2.lve1.5.42.el7.x86_64
    path.separator = :
    sun.arch.data.model = 64
    sun.boot.class.path = /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/resources.jar
        /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/rt.jar
        /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/sunrsasign.jar
        /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/jsse.jar
        /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/jce.jar
        /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/charsets.jar
        /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/jfr.jar
        /home/bitsfetc/virtualenv/java/jre1.8.0_211/classes
    sun.boot.library.path = /home/bitsfetc/virtualenv/java/jre1.8.0_211/lib/amd64
    sun.cpu.endian = little
    sun.cpu.isalist =
    sun.io.unicode.encoding = UnicodeLittle
    sun.java.launcher = SUN_STANDARD
    sun.jnu.encoding = ANSI_X3.4-1968
    sun.management.compiler = HotSpot 64-Bit Tiered Compilers
    sun.os.patch.level = unknown
    user.country = US
    user.dir = /home/bitsfetc/a2np/api
    user.home = /home/bitsfetc
    user.language = en
    user.name = bitsfetc
    user.timezone =

Locale settings:
    default locale = English
    default display locale = English (United States)
    default format locale = English (United States)
    available locales = , ar, ar_AE, ar_BH, ar_DZ, ar_EG, ar_IQ, ar_JO,
        ar_KW, ar_LB, ar_LY, ar_MA, ar_OM, ar_QA, ar_SA, ar_SD,
        ar_SY, ar_TN, ar_YE, be, be_BY, bg, bg_BG, ca,
        ca_ES, cs, cs_CZ, da, da_DK, de, de_AT, de_CH,
        de_DE, de_GR, de_LU, el, el_CY, el_GR, en, en_AU,
        en_CA, en_GB, en_IE, en_IN, en_MT, en_NZ, en_PH, en_SG,
        en_US, en_ZA, es, es_AR, es_BO, es_CL, es_CO, es_CR,
        es_CU, es_DO, es_EC, es_ES, es_GT, es_HN, es_MX, es_NI,
        es_PA, es_PE, es_PR, es_PY, es_SV, es_US, es_UY, es_VE,
        et, et_EE, fi, fi_FI, fr, fr_BE, fr_CA, fr_CH,
        fr_FR, fr_LU, ga, ga_IE, hi, hi_IN, hr, hr_HR,
        hu, hu_HU, in, in_ID, is, is_IS, it, it_CH,
        it_IT, iw, iw_IL, ja, ja_JP, ja_JP_JP_#u-ca-japanese, ko, ko_KR,
        lt, lt_LT, lv, lv_LV, mk, mk_MK, ms, ms_MY,
        mt, mt_MT, nl, nl_BE, nl_NL, no, no_NO, no_NO_NY,
        pl, pl_PL, pt, pt_BR, pt_PT, ro, ro_RO, ru,
        ru_RU, sk, sk_SK, sl, sl_SI, sq, sq_AL, sr,
        sr_BA, sr_BA_#Latn, sr_CS, sr_ME, sr_ME_#Latn, sr_RS, sr_RS_#Latn, sr__#Latn,
        sv, sv_SE, th, th_TH, th_TH_TH_#u-nu-thai, tr, tr_TR, uk,
        uk_UA, vi, vi_VN, zh, zh_CN, zh_HK, zh_SG, zh_TW

java version "1.8.0_211"
Java(TM) SE Runtime Environment (build 1.8.0_211-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)

mktrusty process

$ nanopub-java mktrusty test2/nanopub.trig

this creates a new file (trusty.nanopub.trig), I assume this is the one to check, right?

@prefix this: <http://purl.org/np/RAS1m1Oxr7cjjzavRiqSXuUsZwH2NObA1W5FUjyCY31gU> .
@prefix sub: <http://purl.org/np/RAS1m1Oxr7cjjzavRiqSXuUsZwH2NObA1W5FUjyCY31gU#> .
@prefix authority: <https://hypothes.is/a/> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
@prefix np: <http://www.nanopub.org/nschema#> .
@prefix pav: <http://purl.org/pav/> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

sub:Head {
  this: np:hasAssertion sub:assertion;
    np:hasProvenance sub:provenance;
    np:hasPublicationInfo sub:pubInfo;
    a np:Nanopublication .
}

sub:assertion {
  sub:step dc:description "Centrifuge the tubes (without any delay) for 30 min at 100 x g in 4 °C in a swing-out bucket. Fixed angle rotors also can be used but would require more caution when separating cells in interphase." .
}

sub:provenance {
  sub:assertion pav:createdWith <https://nanotate.bitsfetch.com>;
    prov:generatedAtTime "2021-01-11T11:04:32.705428"^^xsd:dateTime;
    prov:wasAttributedTo <https://hypothes.is/a/acct:miguel.ruano@hypothes.is> .
}

sub:pubInfo {
  sub:assertion prov:wasDerivedFrom authority:E_06qlNvEeuWRR_bAb5r-A .

  this: prov:generatedAtTime "2021-01-11T11:04:32.705428"^^xsd:dateTime;
    prov:wasAttributedTo <https://orcid.org/0000-0002-3784-0922> .
}

check process

$ nanopub-java check test2/trusty.nanopub.trig
Summary: 1 trusty (without signature);

and that's all

tkuhn commented 3 years ago

Can you show what error you get when you run nanopub-java publish test2/trusty.nanopub.trig?

The above nanopublication has a broken trusty URI when I check it. I suspect because of the ° sign. I just checked the code and it always ensures UTF-8 encoding when writing and reading files, so I am not sure why this happens...

miguel-ruano commented 3 years ago

hi @tkuhn the result of executing the command is

$ nanopub-java publish test2/trusty.nanopub.trig
10:15:46,425  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,427  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,428  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,428  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,428  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,428  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,430  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,430  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,431  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,431  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,431  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,432  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,433  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,433  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,433  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,434  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,435  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,435  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,437  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,437  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,437  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,437  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,438  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,438  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,439  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://server.nanopubs.lod.labs.vu.nl:80: Too many open files
10:15:46,440  INFO RetryExec:113 - Retrying request to {}->http://server.nanopubs.lod.labs.vu.nl:80
10:15:46,440  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://server.nanopubs.lod.labs.vu.nl:80: Too many open files
10:15:46,440  INFO RetryExec:113 - Retrying request to {}->http://server.nanopubs.lod.labs.vu.nl:80
10:15:46,441  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://server.nanopubs.lod.labs.vu.nl:80: Too many open files
10:15:46,441  INFO RetryExec:113 - Retrying request to {}->http://server.nanopubs.lod.labs.vu.nl:80
10:15:46,442  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://130.60.24.146:7880: Too many open files
10:15:46,442  INFO RetryExec:113 - Retrying request to {}->http://130.60.24.146:7880
10:15:46,442  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://130.60.24.146:7880: Too many open files
10:15:46,442  INFO RetryExec:113 - Retrying request to {}->http://130.60.24.146:7880
10:15:46,443  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://130.60.24.146:7880: Too many open files
10:15:46,443  INFO RetryExec:113 - Retrying request to {}->http://130.60.24.146:7880
10:15:46,443  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {s}->https://server.nanopubs.knows.idlab.ugent.be:443: Too many open files
10:15:46,444  INFO RetryExec:113 - Retrying request to {s}->https://server.nanopubs.knows.idlab.ugent.be:443
10:15:46,444  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {s}->https://server.nanopubs.knows.idlab.ugent.be:443: Too many open files
10:15:46,444  INFO RetryExec:113 - Retrying request to {s}->https://server.nanopubs.knows.idlab.ugent.be:443
10:15:46,445  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {s}->https://server.nanopubs.knows.idlab.ugent.be:443: Too many open files
10:15:46,445  INFO RetryExec:113 - Retrying request to {s}->https://server.nanopubs.knows.idlab.ugent.be:443
10:15:46,445  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {s}->https://openphacts.cs.man.ac.uk:443: Too many open files
10:15:46,446  INFO RetryExec:113 - Retrying request to {s}->https://openphacts.cs.man.ac.uk:443
10:15:46,446  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {s}->https://openphacts.cs.man.ac.uk:443: Too many open files
10:15:46,446  INFO RetryExec:113 - Retrying request to {s}->https://openphacts.cs.man.ac.uk:443
10:15:46,447  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {s}->https://openphacts.cs.man.ac.uk:443: Too many open files
10:15:46,447  INFO RetryExec:113 - Retrying request to {s}->https://openphacts.cs.man.ac.uk:443
10:15:46,447  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://server.np.scify.org:80: Too many open files
10:15:46,447  INFO RetryExec:113 - Retrying request to {}->http://server.np.scify.org:80
10:15:46,448  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://server.np.scify.org:80: Too many open files
10:15:46,448  INFO RetryExec:113 - Retrying request to {}->http://server.np.scify.org:80
10:15:46,448  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://server.np.scify.org:80: Too many open files
10:15:46,449  INFO RetryExec:113 - Retrying request to {}->http://server.np.scify.org:80
10:15:46,449  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,449  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,450  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,450  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
10:15:46,450  INFO RetryExec:97 - I/O exception (java.net.SocketException) caught when processing request to {}->http://app.tkuhn.eculture.labs.vu.nl:80: Too many open files
10:15:46,450  INFO RetryExec:113 - Retrying request to {}->http://app.tkuhn.eculture.labs.vu.nl:80
java.lang.RuntimeException: Failed to publish the nanopub
        at org.nanopub.extra.server.PublishNanopub.publishNanopub(PublishNanopub.java:217)
        at org.nanopub.extra.server.PublishNanopub.processNanopub(PublishNanopub.java:139)
        at org.nanopub.extra.server.PublishNanopub.access$100(PublishNanopub.java:32)
        at org.nanopub.extra.server.PublishNanopub$1.handleNanopub(PublishNanopub.java:98)
        at org.nanopub.MultiNanopubRdfHandler.finishAndReset(MultiNanopubRdfHandler.java:128)
        at org.nanopub.MultiNanopubRdfHandler.endRDF(MultiNanopubRdfHandler.java:123)
        at org.eclipse.rdf4j.rio.turtle.TurtleParser.parse(TurtleParser.java:201)
        at org.nanopub.MultiNanopubRdfHandler.process(MultiNanopubRdfHandler.java:61)
        at org.nanopub.MultiNanopubRdfHandler.process(MultiNanopubRdfHandler.java:47)
        at org.nanopub.MultiNanopubRdfHandler.process(MultiNanopubRdfHandler.java:53)
        at org.nanopub.extra.server.PublishNanopub.run(PublishNanopub.java:94)
        at org.nanopub.extra.server.PublishNanopub.main(PublishNanopub.java:57)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.nanopub.Run.run(Run.java:76)
        at org.nanopub.Run.main(Run.java:27)
tkuhn commented 3 years ago

OK, thanks, clear!

I overlooked above that you already solved the problems with Java parameters. I added the two utf8 parameters now also in the official start-up script: https://github.com/Nanopublication/nanopub-java/blob/master/bin/np

raar1 commented 3 years ago

Hi @miguel-ruano ,

There are two new releases: fairworkflows, now version 0.2.1 nanopub, now version 1.2.1

I have added the utf8 parameters to this new nanopub version (and updated fairworkflows to use the new version) so this will hopefully solve the current problems in Nanotate. Could you try it out with these new versions and let me know how it goes?

miguel-ruano commented 3 years ago

ok, I will update dependencies and I will tell you the results

miguel-ruano commented 3 years ago

hi @raar1

i did the update and seems works well, the error on publishing does not appear more, thanks for the update !!

raar1 commented 3 years ago

That's great! Let me know if there are any other problems. This issue is pretty long now, so I'm going to close it, but please just open new issues on the fairworkflows or nanopub repos when other stuff shows up :)