How I start my own server listening requests?

[ahojukka5]

https://app.slack.com/client/T68168MUP/C67910KEH/thread/C67910KEH-1575635707.424100

Have an example of how to create resolvers and start a server listening for requests. Is it possible?

[ahojukka5]

Clearly this is not a bug but more like an enhancement for documentation.

[neomatrixcode]

to raise a server with target you can use https://github.com/codeneomatrix/Diana.jl#server

[neomatrixcode]

the myschema object is subsequently the one that receives the request

query= """ { neomatrix{ nombre } } """ result = my_schema.execute(query)

[neomatrixcode]

If you want to have a web server that receives the requests, process them Diana.jl and return the data you can use, for example Merly.jl

https://github.com/codeneomatrix/Merly.jl

[neomatrixcode]

therefore it could have something like:

using Diana
using Merly

schema = Dict(
"query" => "Query"

,"Query"=> Dict(
    "persona"=>Dict("tipo"=>"Persona")
   ,"neomatrix"=>Dict("tipo"=>"Persona")
   )

,"Persona" => Dict(
    "edad"=>Dict("tipo"=>"Int")
   ,"nombre"=>Dict("tipo"=>"String")
  )

)

 resolvers=Dict(
    "Query"=>Dict(
        "neomatrix" => (root,args,ctx,info)->(return Dict("nombre"=>"josue","edad"=>26))
        ,"persona" => (root,args,ctx,info)->(return Dict("nombre"=>"Diana","edad"=>25))
    )
    ,"Persona"=>Dict(
      "edad" => (root,args,ctx,info)->(return root["edad"])
    )
)

my_schema = Schema(schema, resolvers)

server = Merly.app()

#query= """
#{
#neomatrix{
#nombre
#}
#}
#"""

@route POST "/data" begin
  println("body: ",req.body)
  res.headers["Content-Type"]= "application/json"
  my_schema.execute(req.body["query"]) # "{\"data\":{\"neomatrix\":{\"nombre\":\"josue\"}}}"
end

server.start(config=Dict("host" => "127.0.0.1","port" => 8000),verbose=false)

https://codeneomatrix.github.io/Merly.jl/stable/#Dictionary-of-body-1

[neomatrixcode]

At the moment Merly only supports http requests, but in future versions I plan to add more protocols

[neomatrixcode]

there is an implementation of merly for heroku that you can try

https://github.com/codeneomatrix/merly-app Press the button ' Deploy to heroku'

here I have a test of that application working https://pruebaju.herokuapp.com/

[neomatrixcode]

Any comments, questions or problems that may arise, you can create an issue and I will try to respond as soon as possible.

see you later.

[ahojukka5]

Maybe this would be good to add to README.md? I'm considering a workflow where all this stuff is written to Jupyter Notebook and then just throw it to Heroku and there is your backend ready. I guess there are already packages making it possible to run notebooks from initialization scripts.

[ahojukka5]

So there would be Procfile, basically (pseudocode)

web: notebook-runner analysis.ipynb

And there you have it: your scientific analysis results which can be queried from some JavaScript frontend deployed e.g. netlify.com or some similar CDN hosting service. I see a huge potential in this.

[neomatrixcode]

Great, that's a good idea. In the next version of Diana.jl I will create an application for heroku as a demonstration.

Thanks for your comments.

[ahojukka5]

This is almost working. But I get UndefVarError: my_schema not defined from inside of @route macro. I'm also wondering is there coming some problems regarding the http protocol. If I deploy frontend code to e.g. netlify, it will be https. I guess the connection will be failing due to this protocol mismatch..?

[neomatrixcode]

try

global my_schema

it's true http and https are different protocols

[ahojukka5]

Not helping.. Does this have something to do with macro hygiene..?

[neomatrixcode]

It's true, it's because of the macro. This solution works correctly.

using Merly
using Diana

schema = Dict(
"query" => "Query"

,"Query"=> Dict(
    "persona"=>Dict("tipo"=>"Persona")
   ,"neomatrix"=>Dict("tipo"=>"Persona")
   )

,"Persona" => Dict(
    "edad"=>Dict("tipo"=>"Int")
   ,"nombre"=>Dict("tipo"=>"String")
  )

)

 resolvers=Dict(
    "Query"=>Dict(
        "neomatrix" => (root,args,ctx,info)->(return Dict("nombre"=>"josue","edad"=>26))
        ,"persona" => (root,args,ctx,info)->(return Dict("nombre"=>"Diana","edad"=>25))
    )
    ,"Persona"=>Dict(
      "edad" => (root,args,ctx,info)->(return root["edad"])
    )
)

my_schema = Schema(schema, resolvers)

server = Merly.app()

#query= """
#{
#neomatrix{
#nombre
#}
#}
#"""

Post("/data", (req,res)->(begin
  global my_schema
  println("body: ",req.body)
  res.headers["Content-Type"]= "application/json"
  my_schema.execute(req.body["query"]) # "{\"data\":{\"neomatrix\":{\"nombre\":\"josue\"}}}"
end))

server.start(config=Dict("host" => "127.0.0.1","port" => 8000),verbose=false)

I wanted to write "an example in Merly", but I made a mistake. https://codeneomatrix.github.io/Merly.jl/stable/#An-example-in-Diana-1

[ahojukka5]

Yes, now it's working. I'm not sure what are the benefits of using macros when building routes. In schema definition, they definitely would give some syntactic sugar letting you write the schema using the same GraphQL syntax than in JavaScript. Maybe worth trying..?

[ahojukka5]

Another thing which might be worth of considering is to use NamedTuple instead of dictionaries when defining resolvers. I really don't know is this a good idea, but if you don't need all the functionalities provided by dictionaries, like a possibility to modify the resolver on-the-fly, tuples are somewhat simpler structures and might give some performance benefits when doing lookups. With tuples you can also use dot notation making the code maybe look more clear. Consider for example this.

const resolvers = Dict(
    "Query"=>Dict(
        "neomatrix" => (root,args,ctx,info)->(return Dict("nombre"=>"josue","edad"=>26)),
        "persona" => (root,args,ctx,info)->(return Dict("nombre"=>"Diana","edad"=>25))
    ),
    "Persona"=>Dict(
        "edad" => (root,args,ctx,info)->(return root["edad"])
    )
)

const resolvers2 = (
    Query = (
        neomatrix = (root, args, ctx, info) -> (nombre = "josue", edad = 26),
        persona = (root, args, ctx, info) -> (nombre = "Diana", edad = 25)
    ),
    Persona = (
        edad = (root, args, ctx, info) -> root.edad,
    )
)

Then:

julia> resolvers["Query"]["neomatrix"](1,2,3,4)
Dict{String,Any} with 2 entries:
  "edad"   => 26
  "nombre" => "josue"

julia> resolvers2.Query.neomatrix(1,2,3,4)
(nombre = "josue", edad = 26)

We can actually use BenchmarkTools.jl to make some preliminary studies. It looks that resolver defined using tuples might be about 100-200 times faster than the dictionary version (look the number of allocations):

julia> @btime resolvers["Query"]["neomatrix"](1,2,3,4)
  1.249 μs (15 allocations: 1.88 KiB)
Dict{String,Any} with 2 entries:
  "edad"   => 26
  "nombre" => "josue"

julia> @btime resolvers2.Query.neomatrix(1,2,3,4)
  6.990 ns (1 allocation: 32 bytes)
(nombre = "josue", edad = 26)

The reason why the tuple version is so much better performing is that we don't have to make dynamic memory allocations during lookup. And if we use a macro to create a schema, it would be something like

@schema begin
  type Query {
    persona: Persona
    neomatrix: Persona
  }

  type Persona {
    edad: Int
    nombre: String
  }
end

[ahojukka5]

Ok. Now I have some demo numerical simulation set up in Jupyter Notebook. And I also have set up Diana and Merly. But when setting up frontend side, now I'm stuck in CORS:

Access to fetch at 'http://localhost:4000/graphql' from origin 'http://localhost:3000'
has been blocked by CORS policy: Response to preflight request doesn't pass
access control check: It does not have HTTP ok status.

Any ideas how to fix this? Server is build as follows:

server = Merly.app()
server.useCORS(true)
server.start(config=Dict("host" => "127.0.0.1", "port" => 4000), verbose=false)

[neomatrixcode]

Thank you very much for your comments, how are you trying to make your http request? What tool are you using?

[ahojukka5]

Hi, I put everything online. Here's the backend code:

https://github.com/ahojukka5/diana-poc-backend/blob/master/backend.ipynb

And here's the frontend code which cannot access backend for CORS issue:

https://github.com/ahojukka5/diana-poc-frontend/blob/master/src/index.js

For me, everything looks good, but as usual, this is, of course, no guarantee that everything has been done correctly.

[neomatrixcode]

I have reviewed your application with my tools and I have not encountered any problems, so I think you should configure your jupyter. I leave you a few links of readings related to the topic, any problem you may have can leave a message and I will try to help you.

https://stackoverflow.com/questions/42233102/how-to-create-an-insecure-jupyter-server

https://github.com/jupyter/jupyter/issues/79

https://github.com/jupyterhub/jupyterhub/issues/1087

[ahojukka5]

Did you try accessing through the frontend? For me, requesting data using e.g. curl works just fine, but requesting from the frontend is causing problems.

[neomatrixcode]

I could not replicate your error in my tests. Try with

const client = new ApolloClient ({
   uri: 'http: // localhost: 4000 / graphql',
   fetchOptions: {
     mode: 'no-cors',
   },
});

  On the internet I found that it could work.

[ahojukka5]

I actually get an error to notebook:

┌ Error: error handling request
│   exception = (MethodError(getindex, ("{\"operationName\":null,\"variables\":{},\"query\":\"{\\n  solution\\n}\\n\"}", "query"), 0x000000000000690a), Base.StackTraces.StackFrame[(::var"#5#6")(::Merly.myrequest, ::Merly.myresponse) at In[10]:2, handler(::HTTP.Messages.Request) at base.jl:54, handle at Handlers.jl:253 [inlined], handle(::HTTP.Handlers.RequestHandlerFunction{typeof(Merly.handler)}, ::HTTP.Streams.Stream{HTTP.Messages.Request,HTTP.ConnectionPool.Transaction{Sockets.TCPSocket}}) at Handlers.jl:276, #4 at Handlers.jl:345 [inlined], macro expansion at Servers.jl:360 [inlined], (::HTTP.Servers.var"#13#14"{HTTP.Handlers.var"#4#5"{HTTP.Handlers.RequestHandlerFunction{typeof(Merly.handler)}},HTTP.ConnectionPool.Transaction{Sockets.TCPSocket},HTTP.Streams.Stream{HTTP.Messages.Request,HTTP.ConnectionPool.Transaction{Sockets.TCPSocket}}})() at task.jl:333])
└ @ HTTP.Servers /home/jukka/.julia/packages/HTTP/lZVI1/src/Servers.jl:364

It looks a bit that somewhere in the code is string s = "{\"operationName\":null,\"variables\":{},\"query\":\"{\\n solution\\n}\\n\"}" and trying s["query"] fails..?

[neomatrixcode]

In order to help, I must be able to replicate the error on my computer, so I ask you to send me the following information

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. Set up environment
2. Arguments
3. Function/method
4. See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Environment

• Repository
• OS: [e.g. Win/Mac/Linux]
• IDE [e.g. REPL, Jupyter lab/notebook, Aton/Juno, Visual Studio Code]
• Project/Manifest
  • software/package versions

Additional context Add any other context about the problem here.

[neomatrixcode]

OS: Windows 10

IDE: Console

software/package versions Julia: 1.3.0

description of the problem

apollo sends the data to merly with the wrong content-type

Content-Type: text/plain;charset=UTF-8

should be

Content-Type: application/json charset=utf-8

solution

1.- configure the headers from Apollo to be correct

2.- transform data in string format to json from julia, for practicality implement option number two and modify these lines of your original code

using DifferentialEquations, LinearAlgebra, Plots, JSON

Post("/graphql", (req, res) -> begin
  res.headers["Content-Type"]= "application/json"
   if (typeof(req.body)==typeof("string"))
       req.body=JSON.parse(req.body)
   end
  my_schema.execute(req.body["query"])
end)

I hope this can help you

[neomatrixcode]

After doing more tests I ran into the following error: Unexpected end of JSON

after much reading, this is due to the way in which Apollo works so it is necessary to remove the instruction fetchOptions: {       mode: 'no-cors',     },

and add an OPTIONS method in addition to an extra "Access-Control-Allow-Headers" header.

Post("/graphql", (req, res) -> begin
  res.headers["Access-Control-Allow-Headers"]= "*"
  res.headers["Content-Type"]= "application/json"
  my_schema.execute(req.body["query"])
end)

@route OPTIONS "/graphql" begin
  res.headers["Access-Control-Allow-Headers"]= "*"
  res.headers["Content-Type"]= "text/plain"
  ""
end

in the following versions of Merly I will add the native support to Apollo so that it is not necessary to add these elements, in addition to creating a test application for heroku to serve as a guide for upcoming implementations.

[neomatrixcode]

Thank you very much for your comments