"TypeError: Cannot read property 'length' of undefined"

[jakebeal]

When attempting to browse https://hub.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/1 , it stalls for a long time before failing with a web page reading:

Error
TypeError: Cannot read property 'length' of undefined
    at componentDefinition.sequenceAnnotations.sort (/data/synbiohub/node_modules/visbol/lib/getDisplayList.js:348:21)
    at sort (native)
    at Array.sort (native)
    at sortedSequenceAnnotations (/data/synbiohub/node_modules/visbol/lib/getDisplayList.js:346:52)
    at getDisplayListSegment (/data/synbiohub/node_modules/visbol/lib/getDisplayList.js:190:19)
    at getDisplayList (/data/synbiohub/node_modules/visbol/lib/getDisplayList.js:14:9)
    at renderView (/data/synbiohub/lib/views/componentDefinition.js:457:35)
    at process._tickCallback (internal/process/next_tick.js:109:7)

[cjmyers]

This object appears to be very strange. It does not appear to refer to valid SBOL. It looks like it might have been generated by our old JBEI GenBank to SBOL converter, but I’m not sure. It has a lot of SequenceAnnotations that appear to not be properly formatted. This is clearly causing fits to VisBOL that expects valid SBOL. However, it is fair to say it should be robust to invalid files and fail more gracefully.

On Feb 1, 2019, at 12:52 PM, Jacob Beal notifications@github.com wrote:

When attempting to browse https://hub.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/1 https://hub.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/1 , it stalls for a long time before failing with a web page reading:

Error TypeError: Cannot read property 'length' of undefined at componentDefinition.sequenceAnnotations.sort (/data/synbiohub/node_modules/visbol/lib/getDisplayList.js:348:21) at sort (native) at Array.sort (native) at sortedSequenceAnnotations (/data/synbiohub/node_modules/visbol/lib/getDisplayList.js:346:52) at getDisplayListSegment (/data/synbiohub/node_modules/visbol/lib/getDisplayList.js:190:19) at getDisplayList (/data/synbiohub/node_modules/visbol/lib/getDisplayList.js:14:9) at renderView (/data/synbiohub/lib/views/componentDefinition.js:457:35) at process._tickCallback (internal/process/next_tick.js:109:7) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SynBioHub/synbiohub/issues/808, or mute the thread https://github.com/notifications/unsubscribe-auth/ADWD99hBV-ZuQEDr6_5Ui2wIwtsQu_Vfks5vJI0DgaJpZM4ae4f-.

[jakebeal]

Is the old JBEI converter accessible online? Perhaps somebody found it and used it?

[jakebeal]

Can you tell if anything is actually using the file? It might be best for us to simply delete it, if I can confirm with Mark that it's not used with any data either.

[jakebeal]

@cjmyers: this part isn't being used in anything that Mark has received, and so I've deleted it from the SD2 Dictionary. Can you please excise it from the SD2 SynBioHub instance as well, after you've learned what you need to learn about the error?

[cjmyers]

It is used in these three collections:

https://hub.sd2e.org/user/sd2e/design/design_collection/1 https://hub.sd2e.org/user/sd2e/design/novel_chassis_genomes/1 https://hub.sd2e.org/user/sd2e/design/novel_chassis/1

This query will tell you this:

select ?s ?p where { ?s ?p https://hub.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/1 }

At: https://hub.sd2e.org/sparql

Make sure to select "My Submissions"

This query will show you what it includes:

select ?p ?o where { https://hub.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/1 ?p ?o }

I was wrong. It was generated by our GenBank converter. It is a fairly big construct with a lot of sequence annotations. I do want to try to sort out why it is not fetching. We also need to fix up the VisBOL rendering to be more fail safe in these cases.

[jakebeal]

Being contained in those collections doesn't matter if it's not being pointed to other than that, so it can be deleted (I've already deleted it from the dictionary). Do please continue your forensics, however.

[cjmyers]

Somehow a number of the sequence annotations for this object are not showing up in Virtuoso (see part of log below). This causes a non-local fetch to be attempted, which really hits this repository again and tries to reload the entire object (see it is a child). This fails, since it is not present. This is clearly a waste of effort, so these non-local fetches should never be attempted for a local and/or child object fetch. See issue #815.

Furthermore, since these are unable to be found, VisBOL is given an incomplete SBOL object that is missing child objects. This should never happen, so VisBOL is not prepared for it and crashes trying to dereference a pointer of an undefined object. VisBOL clearly should have better error checking for such invalid files, but in the meantime, we should catch VisBOL errors and instead of failing to render the page, just fail to render the VisBOL image. See issue #816.

Finally, this leaves the question as to why these objects are not in the data store. That one is harder to work out. I don't have a great hypothesis for this one. The only thing I can think of is that it somehow got interrupted while uploading this large object to VIrtuoso. Some SequenceAnnotations are there, but not all of them. Making submissions atomic (see issue #810) would eliminate this risk, assuming that is what caused this state. Another possibility might be that a removal of this object was interrupted, which again implies that removals should be atomic. In both of these cases, the Virtuoso updates are indeed atomic, at least as far as we know. However, some Virtuoso operations need to be split up or Virtuoso crashes. One example is delete which is done in groups of 10000 triples at a time. One idea might be to delete critical bits in the first step which may make an object inaccessible with potential downside of spurious triples left. Another idea might be to have a means to detect invalid SBOL objects within the triple store and provide a means to clean these out periodically.

In any case, would like to leave this record for the time being to enable a test for #815 and #816, and maybe a future cleanup issue.

19 URI(s) left to resolve Fetching non-local:https://hub-staging.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/annotation2602/1/sbol Fetching non-local:https://hub-staging.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/annotation2198/1/sbol Fetching non-local:https://hub-staging.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/annotation258/1/sbol Fetching non-local:https://hub-staging.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/annotation2189/1/sbol Fetching non-local:https://hub-staging.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/annotation2575/1/sbol Fetching non-local:https://hub-staging.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/annotation2113/1/sbol Fetching non-local:https://hub-staging.sd2e.org/user/sd2e/design/Strain_5_MG1655_Genomic_pBADmin/annotation2552/1/sbol

[cjmyers]

This page is now opening on staging after a bit of delay. Want to keep this open though a bit longer to have a look at it when PR #826 is merged.

[cjmyers]

This object now opens relatively quickly. Last step will be to delete this object, if it is unneeded.

[jakebeal]

Please do delete it. If somebody needs it in the future, it can be recreated with better formatting.

[cjmyers]

I was able to successfully remove it from staging, but I will need to wait for production to be updated before I can remove it from there.

[jakebeal]

OK, then let's leave this open until then --- staging will, of course, get wiped by the backups until that time.

[jakebeal]

@cjmyers Is this resolved in the new deployment?

[cjmyers]

Yes, this record is now opening up. However, do you still want me to remove it?

On Feb 26, 2019, at 8:12 AM, Jacob Beal notifications@github.com wrote:

@cjmyers https://github.com/cjmyers Is this resolved in the new deployment?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SynBioHub/synbiohub/issues/808#issuecomment-467476587, or mute the thread https://github.com/notifications/unsubscribe-auth/ADWD96P7ivTnMBAlaAyMMuMcBw4HJhp9ks5vRU76gaJpZM4ae4f-.

[jakebeal]

Yes, please. The record may be opening, but it's full of potential problems and nobody is using it. Please delete it, and if somebody needs it again in the future, it can be recreated in a cleaner form.

[cjmyers]

It has been deleted.