.json record containing a value not in the schema

[jpk96]

@vkuznet:

When submitting record (i.e. .json file) from SPEC: Even if a json contains a value that is not in the list of values in the schema, the record gets accepted to dev server. For example, in /nfs/chess/aux/metadata/cycles/2023-3/id1a3/testuser-1111-a/test-1/45/All.json, "Beamline" has a value of ["id1a3"] , rather than ["1A3"] (this was done on purpose for testing). Is this behavior expected? Is there a way to catch an error where .json contains a value in a list which is not in the corresponding list of values in the schema?

[vkuznet]

This now should be fixed on dev cluster, see 906f413baed7dbdaba2ae7666ed26e47566e646c Please re-run your tests and confirm, then I can propagate changes to production server.

[jpk96]

While attempting to run tests by alternating between submitting a record to the dev server with an invalid entry for "Beamline" in one case and a valid entry for the same key in another, I have run into an error that I have not seen before:

On id1a3 station computer, the following error appears when submitting a json with valid values /nfs/chess/aux/metadata/cycles/2023-3/id1a3/testuser-1111-a/test-1/99/All.json:

panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x694296]

goroutine 1 [running]: main.placeRequest({0x7fff2d932f05, 0x5}, {0x7fff2d932f3b, 0x29}, {0x7fff2d932eae?, 0x70f128?}, {0x7fff2d932f14, 0x21}, 0x2) /nfs/chess/user/rv43/ChessDataManagement/client/chess_client.go:220 +0x636 main.main() /nfs/chess/user/rv43/ChessDataManagement/client/chess_client.go:357 +0x468

On id3a station computer: The following error appears when submitting a json with valid values /nfs/chess/aux/metadata/cycles/2023-3/id3a/testuser-1111-a/test-1/36/All.json:

panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x694296]

goroutine 1 [running]: main.placeRequest({0x7fffab27efa7, 0x4}, {0x7fffab27efdb, 0x29}, {0x7fffab27ef51?, 0x70f128?}, {0x7fffab27efb5, 0x20}, 0x1) /nfs/chess/user/rv43/ChessDataManagement/client/chess_client.go:220 +0x636 main.main() /nfs/chess/user/rv43/ChessDataManagement/client/chess_client.go:357 +0x468

Error messages from id1a3 and id3a look very similar but not quite identical.

[jpk96]

Trying this a few minutes later with /nfs/chess/aux/metadata/cycles/2023-3/id1a3/testuser-1111-a/test-1/101/All.json, the following error message is shown:

{"error":"invalid data value for key=SampleState, type=list_str, multiple=false, value=[Powder]","status":400}, error 2023/09/26 20:28:17 chess_client.go:83: request fails with status: 400 Bad Request, error:

In that .json, the value for SampleState is ["Powder"], which is valid. The error message appears to suggest that there are no double quotation marks for the string in the list.

[vkuznet]

The issue is that we have extra empty space in SampleState schema value for Powder, like this "Powder ", see https://github.com/vkuznet/ChessDataManagement/blob/master/web/schemas/ID1A3.json#L796. So, the code is correct that you provided "Powder" while schema has "Powder ". I think nothing needs to be done here and instead you should correct the schema to remove this empty space :)

[vkuznet]

I already fixed "Powder" issue along with invalid memory address or nil pointer dereference. I was able to successfully inject All.json from your location using ID1A3 schema. Could you please try again and close the issue if everything will be fine on your side.

[rolfverberg]

I rebuild the chess_client executable on /nfs/chess/sw/chessdata when I had an issue before. I notice the rv43 path in the panic error message. The chess_client may also need rebuilding.

[vkuznet]

It is not a chess_client as error came from MetaData server itself, therefore no need to rebuild the client, and I tested new code with /nfs/chess/sw/chessdata/chess_client and it works for me.

[rolfverberg]

Perfect. I was able to submit Peter's json files without error just now.

This one: /nfs/chess/aux/metadata/cycles/2023-3/id1a3/testuser-1111-a/test-1/99/All.json

And this one as well: /nfs/chess/aux/metadata/cycles/2023-3/id3a/testuser-1111-a/test-1/36/All.json

[jpk96]

Thanks all. I was able to test this.

Submitting /nfs/chess/aux/metadata/cycles/2023-3/id1a3/testuser-1111-a/test-1/107/All.json causes an error because "Beamline" key is submitted as ["id1a3"], and "id1a3" is not in the list of strings for the key in ID1A3 schema.

Submitting /nfs/chess/aux/metadata/cycles/2023-3/id1a3/testuser-1111-a/test-1/108/All.json was successful, because "Beamline": key is submitted as ["1A3"].

I will close this issue.