some IDs make create-chant break?

[annamorphism]

I just got an email from an indexer who had created the first chant in a source, but got a 500 error when trying to add a second. I tried myself and also can't seem to add a new chant to https://cantusdatabase.org/source/1000051.

When I created a chant with identical information in a test source, it also gave me a 500 error, but added the chant to the source. When I then tried to add new chants to the test source, it gave a 500. Remove the new chant--> can create as normal. I tried removing the content in various fields in the new chant, and strangely, deleting or replacing the Cantus ID seemed to solve the problem. Why would the ID g00791 be making it so you can't create a new chant in that source? Further investigation required.

[dchiller]

I suspect this will be an error in the process of getting suggested next chants. That's not supposed to break anything if it fails (it's just supposed to show you no suggestions), but I think there is some way it is here... will investigate.

[annamorphism]

That was my suspicion too! but it's very weird that it seems to happen on this specific and otherwise ordinary-seeming ID...

[dchiller]

Ok... this is interesting.

When the suggested chants function starts looking for the next chants after g00791, Cantus Index responds with the following top three Cantus ID's:

• 'g00791a '
• 'g00711'
• 'g01957'

Note that the first ID has a trailing space.

When the function then asks for the data on each of these ID's, the inclusion of the space in the case of the first ID causes Cantus Index to respond with all of the chants in the Cantus network with that Cantus ID (in other words, all the concordances) but crucially not the metadata for that Cantus ID (i.e. the full text, the genre, etc.) that we use to populate the chant suggestion on CantusDB.

So:

1. Cantus Index is returning that slightly mal-formed Cantus ID, and then
2. It can use that Cantus ID to return a response (and thus not triggering CantusDB's built-in error handling) but not the elements of the response we are expecting (thus triggering a 500 error).

We should make our suggested chants feature more robust to this error. But also, what's going on with this CantusID in Cantus Index?

[dchiller]

But also, what's going on with this CantusID in Cantus Index?

@annamorphism Do you have access to see whether this spacing was perhaps an entry error in Cantus Index?

[annamorphism]

But also, what's going on with this CantusID in Cantus Index?

@annamorphism Do you have access to see whether this spacing was perhaps an entry error in Cantus Index?

I looked, and there doesn't seem to be trailing space, but maybe it's somehow cleaned up before I can interact with it. I tried opening and saving it--does that make it go away? I imagine there could be other examples of this sort of at random in the index, generated by human error when somebody manually changed an ID.

[annamorphism]

I found another ten or so trailing whitespace examples and fixed them. They were more obvious cases than this one, though, so I don't know if there are others lurking somewhere...

[dchiller]

The other weird thing is that it seems like the Cantus Index API we are currently using (json-nextchants) is only available on the new Cantus Index site (cantusindex.uwaterloo.ca) so there also may be something specific going on there).

[annamorphism]

on New Cantus Index https://cantusindex.uwaterloo.ca/id/g00791a doesn't look like it has a trailing space (unlike say https://cantusindex.uwaterloo.ca/id/008367%3A03), and I don't think it's from my meddling earlier this afternoon (since it purports to have been last modified in 2016). So the trailing space is coming in through the API, but not obviously present on Cantus Index itself...

[dchiller]

So the trailing space is coming in through the API, but not obviously present on Cantus Index itself...

Yes... I wondered if it had to do with Cantus ID's that end with letters and that particular API, but https://cantusindex.uwaterloo.ca/json-nextchants/g00791 returns g00791f just fine!

I am going to alert Jan to this since we've "fixed" it in CantusDB (we just won't show suggestions when this happens) and it doesn't seem to be an issue with the underlying data. I'll cc you both.

I think we could probably close this issue since #1517 did everything necessary on our end. @annamorphism I don't know if you want to communicate with the user that this will be solved on production shortly, or otherwise keep this issue open for any reason? I think they will just have to move on to other chants and come back to this one when we have it up on production.

[annamorphism]

@dchiller that's basically the case now--that chant just has no ID and they moved on to doing something else for now. Once it's fixed on production I'll send another note. Closing issue.

[dchiller]

Jan wrote today that this has been fixed...and I just confirmed that Cantus ID g00791 works