Set up the JavaScript library following installation steps in the Readme

[Georjane]

all tests passing for the spectral-workbench.js setup

101 specs in 174.257s.
>> 0 failures

Done.

@jywarren I tried writing this example script from the spectral-workbench.js readme and manipulating the spectrum data, which ran successfully except for the experimental direct upload feature, which gave me a response of {"status":406,"error":"Not Acceptable"}

#!/usr/bin/env node

var SpectralWorkbench = require('spectral-workbench').SpectralWorkbench;

var data = {
  "data": { 
    "name": "Test spectrum",
    "lines":[ // as many data points as you like may be entered here:
      {"average":64.3333,"r":69,"g":46,"b":78,"wavelength":269.089},
      {"average":63.3333,"r":71,"g":45,"b":74,"wavelength":277.718},
      {"average":64,"r":71,"g":47,"b":74,"wavelength":291.524},
      {"average":64,"r":68,"g":49,"b":75,"wavelength":303.604}
    ]
  }
}

var spectrum = new SpectralWorkbench.Spectrum(data);
spectrum.addAndParseTag('smooth:3');
console.log(spectrum.getIntensity(282));

spectrum.upload(
  'https://spectralworkbench.org/spectrums.json', 
  function callback(err, httpResponse, body) { console.log(body) }, 
  { title: "Jane testing" },
  { token: "xxxxxxxxx"},
);

$ node index.js
      parsing smooth:3
      0.10571428571428572
      {"status":406,"error":"Not Acceptable"}

[jywarren]

Ah, i wonder if that route has also gotten broken? i'm glad you were able to get that far in generating a spectrum on the commandline!

I wonder if it's due to some change in how Rails reads JSON submissions. It's not a typical error like a 500 or a 404 or something. A 406 may be linked to the formatting of the submission:

https://stackoverflow.com/questions/3751030/rails-3-returning-a-http-406-not-acceptable https://github.com/rails-api/active_model_serializers/issues/2133

These aren't super helpful except that they hint at a link between 406 errors and JSON. But the best way to debug this might be to run the Rails app locally too -- for example, you could run it at the usual Rails port 3000, so http://localhost:3000, and then you could run your spectrum.upload() code pointing at http://localhost:3000/spectrums.json. That way you'll be able to see "behind the scenes" of the Rails app too, and see what the exact rails error log is.

How does that sound? Thanks @Georjane !!!

[jywarren]

Oh one more! https://coolaj86.com/articles/rails-error-406.html

[Tlazypanda]

On further investigating locally I think the token and title params are not getting passed correctly since it's entering the else condition ln 305 (spectrums_controller.rb) where the total body is not considered to be a json format hence the error

Note: In this I am passing the title like this in data since it is being called as params[:spectrum][:title] in the controller

var data = {
  "title":"uploaded data",
  "data": { 
    "name": "Test spectrum",
    "lines": JSON.parse(x)
  }
}

Started POST "/spectrums.json" for 127.0.0.1 at 2021-12-16 12:58:06 +0530
Processing by SpectrumsController#create as JSON
  Parameters: {"spectrum"=>{"title"=>"uploaded data", "data_type"=>"json", "data"=>"[{\"average\":64.3333,\"r\":69,\"g\":46,\"b\":78,\"wavelength\":269.089},{\"average\":63.3333,\"r\":71,\"g\":45,\"b\":74,\"wavelength\":277.718},{\"average\":64,\"r\":71,\"g\":47,\"b\":74,\"wavelength\":291.524},{\"average\":64,\"r\":68,\"g\":49,\"b\":75,\"wavelength\":303.604}]"}, "token"=>"31363339363334343530"}
  User Load (0.4ms)  SELECT  `users`.* FROM `users` WHERE (created_at > '2021-12-16 06:00:49' AND created_at < '2021-12-16 06:00:51') ORDER BY `users`.`id` ASC LIMIT 1
  ↳ app/models/user.rb:117
Completed 406 Not Acceptable in 2ms (ActiveRecord: 0.4ms)

I am still unsure since running this scripts/upload.js --file test.json --token <my_token> works fine on spectralworkbench.org but for localhost fails and gives the same error although the files haven't been modified for upload.js script or the controller script cc @jywarren @Georjane

[Georjane]

Ah, i wonder if that route has also gotten broken? i'm glad you were able to get that far in generating a spectrum on the commandline!

I wonder if it's due to some change in how Rails reads JSON submissions. It's not a typical error like a 500 or a 404 or something. A 406 may be linked to the formatting of the submission:

https://stackoverflow.com/questions/3751030/rails-3-returning-a-http-406-not-acceptable rails-api/active_model_serializers#2133

These aren't super helpful except that they hint at a link between 406 errors and JSON. But the best way to debug this might be to run the Rails app locally too -- for example, you could run it at the usual Rails port 3000, so http://localhost:3000, and then you could run your spectrum.upload() code pointing at http://localhost:3000/spectrums.json. That way you'll be able to see "behind the scenes" of the Rails app too, and see what the exact rails error log is.

How does that sound? Thanks @Georjane !!!

Thanks @jywarren I did go through the articles you share. They were very helpful. I think the problem I had was the format of my data. That has being fixed. Right now I have the same error message as @Tlazypanda

Started POST "/spectrums.json" for 127.0.0.1 at 2021-12-17 14:50:25 +0100
DEPRECATION WARNING: Using a dynamic :action segment in a route is deprecated and will be removed in Rails 6.0. (called from block in <top (required)> at /home/jane/outreachy/spectral-workbench/config/routes.rb:195)
Processing by SpectrumsController#create as JSON
  Parameters: {"spectrum"=>{"title"=>"Jane spectrum", "data_type"=>"json", "data"=>"[{\"average\":64.3333,\"r\":69,\"g\":46,\"b\":78,\"wavelength\":269.089},{\"average\":63.3333,\"r\":71,\"g\":45,\"b\":74,\"wavelength\":277.718},{\"average\":64,\"r\":71,\"g\":47,\"b\":74,\"wavelength\":291.524},{\"average\":64,\"r\":68,\"g\":49,\"b\":75,\"wavelength\":303.604}]"}, "token"=>"xxxxxxxxxxxx"}
  User Load (0.3ms)  SELECT  `users`.* FROM `users` WHERE (created_at > '2021-11-02 06:56:00' AND created_at < '2021-11-02 06:56:02') ORDER BY `users`.`id` ASC LIMIT 1
  ↳ app/models/user.rb:117
App 3265 output: outttt
Completed 406 Not Acceptable in 29ms (ActiveRecord: 5.6ms)

[Georjane]

Ah, i wonder if that route has also gotten broken? i'm glad you were able to get that far in generating a spectrum on the commandline!

I wonder if it's due to some change in how Rails reads JSON submissions. It's not a typical error like a 500 or a 404 or something. A 406 may be linked to the formatting of the submission:

https://stackoverflow.com/questions/3751030/rails-3-returning-a-http-406-not-acceptable rails-api/active_model_serializers#2133

These aren't super helpful except that they hint at a link between 406 errors and JSON. But the best way to debug this might be to run the Rails app locally too -- for example, you could run it at the usual Rails port 3000, so http://localhost:3000, and then you could run your spectrum.upload() code pointing at http://localhost:3000/spectrums.json. That way you'll be able to see "behind the scenes" of the Rails app too, and see what the exact rails error log is.

How does that sound? Thanks @Georjane !!!

Thanks @jywarren I did go through the articles you share. They were very helpful. I think the problem I had was the format of my data. That has being fixed. Right now I have the same error message as @Tlazypanda

Started POST "/spectrums.json" for 127.0.0.1 at 2021-12-17 14:50:25 +0100
DEPRECATION WARNING: Using a dynamic :action segment in a route is deprecated and will be removed in Rails 6.0. (called from block in <top (required)> at /home/jane/outreachy/spectral-workbench/config/routes.rb:195)
Processing by SpectrumsController#create as JSON
  Parameters: {"spectrum"=>{"title"=>"Jane spectrum", "data_type"=>"json", "data"=>"[{\"average\":64.3333,\"r\":69,\"g\":46,\"b\":78,\"wavelength\":269.089},{\"average\":63.3333,\"r\":71,\"g\":45,\"b\":74,\"wavelength\":277.718},{\"average\":64,\"r\":71,\"g\":47,\"b\":74,\"wavelength\":291.524},{\"average\":64,\"r\":68,\"g\":49,\"b\":75,\"wavelength\":303.604}]"}, "token"=>"xxxxxxxxxxxx"}
  User Load (0.3ms)  SELECT  `users`.* FROM `users` WHERE (created_at > '2021-11-02 06:56:00' AND created_at < '2021-11-02 06:56:02') ORDER BY `users`.`id` ASC LIMIT 1
  ↳ app/models/user.rb:117
App 3265 output: outttt
Completed 406 Not Acceptable in 29ms (ActiveRecord: 5.6ms)

[jywarren]

OK, so let's look carefully at the exact full request, which as Rails reports in your last comment, would be:

{"spectrum"=>{"title"=>"Jane spectrum", "data_type"=>"json", "data"=>"[{\"average\":64.3333,\"r\":69,\"g\":46,\"b\":78,\"wavelength\":269.089},{\"average\":63.3333,\"r\":71,\"g\":45,\"b\":74,\"wavelength\":277.718},{\"average\":64,\"r\":71,\"g\":47,\"b\":74,\"wavelength\":291.524},{\"average\":64,\"r\":68,\"g\":49,\"b\":75,\"wavelength\":303.604}]"}, "token"=>"xxxxxxxxxxxx"}

As @Tlazypanda notes, maybe we need to look carefully at how it's being converted from JSON and accessed in Rails. Let's link in an excerpt of the exact controller method that's reading it, which from your log is SpectrumsController#create as JSON:

https://github.com/publiclab/spectral-workbench/blob/8bbbe257a5426cbdc7c9171ee08b55b91a72d1db/app/controllers/spectrums_controller.rb#L202-L228

We can already see it's trying to access params[:token] directly, which pulls from the GET or POST parameters. Is it successfully getting a token? Can we log out puts params[:token] to check?

I see we also have data_type: "json" which means we are getting into the conditional on line 223:

https://github.com/publiclab/spectral-workbench/blob/8bbbe257a5426cbdc7c9171ee08b55b91a72d1db/app/controllers/spectrums_controller.rb#L223-L225

Let's go through each line and see what's happening, and if it's really getting the correct JSON data for each use of params[].

Interestingly, down on this line we say to return an error which corresponds to 422, not 406, i think?

https://github.com/publiclab/spectral-workbench/blob/8bbbe257a5426cbdc7c9171ee08b55b91a72d1db/app/controllers/spectrums_controller.rb#L289

Aha - here are a couple posts that seem pretty relevant!

• https://stackoverflow.com/questions/1414446/weird-406-not-acceptable-error
• https://stackoverflow.com/questions/4662369/rails-3-406-not-acceptable-error-on-attempting-to-use-format-js-cant-change-co

There are a few ways to manage "format" or type, i.e. detecting if it's JavaScript sending something, if it's an XHR request, if the request ends in .json, etc. and I believe the conventions on this have changed over the years in Rails. We may be mistaken in how we're trying to get the code to recognize that we're dealing with a JS request, in JSON format. I think the first of the above links is the best place to look!

Hope this helps, @Georjane -- thanks for sticking with this!

[jywarren]

And just for context - i think this direct means of uploading a spectrum is important -- but is not the way the JS library typically does it while running within the Rails app. This is more like a purely command-line utility which not too many people use. That's probably why it was broken and nobody noticed?

[Georjane]

Ah, i wonder if that route has also gotten broken? i'm glad you were able to get that far in generating a spectrum on the commandline!

I wonder if it's due to some change in how Rails reads JSON submissions. It's not a typical error like a 500 or a 404 or something. A 406 may be linked to the formatting of the submission:

https://stackoverflow.com/questions/3751030/rails-3-returning-a-http-406-not-acceptable rails-api/active_model_serializers#2133

These aren't super helpful except that they hint at a link between 406 errors and JSON. But the best way to debug this might be to run the Rails app locally too -- for example, you could run it at the usual Rails port 3000, so http://localhost:3000, and then you could run your spectrum.upload() code pointing at http://localhost:3000/spectrums.json. That way you'll be able to see "behind the scenes" of the Rails app too, and see what the exact rails error log is.

How does that sound? Thanks @Georjane !!!

Thanks @jywarren I did go through the articles you share. They were very helpful. I think the problem I had was the format of my data. That has being fixed. Right now I have the same error message as @Tlazypanda

Started POST "/spectrums.json" for 127.0.0.1 at 2021-12-17 14:50:25 +0100
DEPRECATION WARNING: Using a dynamic :action segment in a route is deprecated and will be removed in Rails 6.0. (called from block in <top (required)> at /home/jane/outreachy/spectral-workbench/config/routes.rb:195)
Processing by SpectrumsController#create as JSON
  Parameters: {"spectrum"=>{"title"=>"Jane spectrum", "data_type"=>"json", "data"=>"[{\"average\":64.3333,\"r\":69,\"g\":46,\"b\":78,\"wavelength\":269.089},{\"average\":63.3333,\"r\":71,\"g\":45,\"b\":74,\"wavelength\":277.718},{\"average\":64,\"r\":71,\"g\":47,\"b\":74,\"wavelength\":291.524},{\"average\":64,\"r\":68,\"g\":49,\"b\":75,\"wavelength\":303.604}]"}, "token"=>"xxxxxxxxxxxx"}
  User Load (0.3ms)  SELECT  `users`.* FROM `users` WHERE (created_at > '2021-11-02 06:56:00' AND created_at < '2021-11-02 06:56:02') ORDER BY `users`.`id` ASC LIMIT 1
  ↳ app/models/user.rb:117
App 3265 output: outttt
Completed 406 Not Acceptable in 29ms (ActiveRecord: 5.6ms)

[jywarren]

Ah, maybe i misunderstood - if you have fixed the data format issue, why do you think you're still seeing the 406 error? I think a similar approach as I had mentioned - using rails log statements to see what happens at each step - could help diagnose this too. Let's see on what line it actually generates the 406 error?