Parse.Cloud.startJob not returning jobStatusId

nebitrams commented 9 months ago

New Issue Checklist

[x] I am not disclosing a vulnerability.
[x] I am not just asking a question.
[x] I have searched through existing issues.
[x] I can reproduce the issue with the latest version of Parse Server.

Issue Description

I called Parse.Cloud.startJob in a cloud function. I expect Parse.Cloud.startJob to return jobStatusId but I received an empty dictionary. This is the specification of Parse.Cloud.startJob, http://parseplatform.org/Parse-SDK-JS/api/4.0.1/Parse.Cloud.html#.startJob

This issue only occur after I upgrade parse-server from 5.4.0 -> 6.3.0

Is this a bug or change in Parse.Cloud.startJob behaviour?

Steps to reproduce

Define a job and start the job in a cloud function. Since I am building iOS App, I call the startTestJob using Parse-SDK-iOS-OSX library.

Parse.Cloud.define("startTestJob", async (request) => {
  var jobId = await Parse.Cloud.startJob('testJob', request.params);
  return jobId;
}); 

Parse.Cloud.job("testJob", async (request) =>  {
  return "testJob Done";
});

iOS Code

PFCloud.callFunction(inBackground: "startTestJob", withParameters:[:]) {
            (result: Any?, error: Error?) -> Void in
...
}

Actual Outcome

The result is empty dictionary.

info: Ran cloud function startTestJob for user XYZ with:
  Input: {}
  Result: {} {"functionName":"startTestJob","params":{},"user":"YVNLpZY6k6"}
verbose: RESPONSE from [POST] /parse/functions/startTestJob: {
  "response": {
    "result": {}
  }
} {"result":{"response":{"result":{}}}}

Expected Outcome

I expect the result to contain the jobStatusID.

FYI, I tried another way to start the job. Starting the job from the parse dashboard website, I see the jobStatusId in the server verbose log.

verbose: REQUEST for [POST] /parse/jobs: {
  "description": "Executing from job schedule web console.",
  "input": {},
  "jobName": "testJob",
  "when": 0
} {"body":{"description":"Executing from job schedule web console.","input":{},"jobName":"testJob","when":0},"headers":{"accept":"*/*","accept-encoding":"gzip, deflate","accept-language":"en-SG,en-GB;q=0.9,en;q=0.8","connection":"keep-alive","content-length":"298","content-type":"text/plain","host":"localhost:1337","origin":"http://localhost:4040","referer":"http://localhost:4040/","sec-fetch-dest":"empty","sec-fetch-mode":"cors","sec-fetch-site":"same-site","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15"},"method":"POST","url":"/parse/jobs"}
verbose: RESPONSE from [POST] /parse/jobs: {
  "headers": {
    "X-Parse-Job-Status-Id": "47QXAcrSaN"
  },
  "response": {}
} {"result":{"headers":{"X-Parse-Job-Status-Id":"47QXAcrSaN"},"response":{}}}

Environment

Server

Parse Server version: 6.3.0
Operating system: MacOS Ventura 13.5.1
Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): Local

Database

System (MongoDB or Postgres): Mongo
Database version: 6.0.4
Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): Local

Client

SDK (iOS, Android, JavaScript, PHP, Unity, etc): Parse-SDK-iOS-OSX
SDK version: 2.7.0

Logs

parse-github-assistant[bot] commented 9 months ago

Thanks for opening this issue!

🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

mtrezza commented 9 months ago

Can you try to call the REST API directly and see what it returns?

nebitrams commented 9 months ago

I ran against the REST API directly using curl. The same problem remains. See the result in the second line below.

From the server VERBOSE log, I can see the jobStatusId in the response header but the body remain as empty.

% ./run_startJob_local.sh 
{}
% cat run_startJob_local.sh 
curl -k -X POST \
  -H "X-Parse-Application-Id: app-key" \
  -H "X-Parse-REST-API-Key: rest-key" \
  -H "X-Parse-Master-Key: master-key" \
  -H "Content-Type: application/json" \
  http://localhost:1337/parse/jobs/testJob \
  -H "X-Parse-Session-Token: session-token"

See the server VERBOSE log.

verbose: RESPONSE from [POST] /parse/jobs/testJob: {
  "headers": {
    "X-Parse-Job-Status-Id": "hogbOlmyJe"
  },
  "response": {}
} {"result":{"headers":{"X-Parse-Job-Status-Id":"hogbOlmyJe"},"response":{}}}

mtrezza commented 9 months ago

What is the response of the curl, does it have a response header "X-Parse-Job-Status-Id? I doubt that the job status should be in the body if Parse.Cloud.Job allows to return a custom response, because that would be a conflict.

nebitrams commented 9 months ago

The curl response is as follows. See the second line.

% ./run_startJob_local.sh 
{}

I can't see the response header from the curl command line. However, I see the response header with Job-Status-Id in the server side verbose log.

I think the old (5.4.0) parse-server API return the jobStatusId in the body. My app works well before I upgraded the server to parse-server 6.3.0.

Is there a change in the API interface in version 6? I am not aware of the custom response that you mentioned.

Also, from the Parse-SDK-iOS-OSX SDK, how do I get the jobStatusId? Apparently, the API only retrieve the result from the body.

PFCloud.callFunction(inBackground: "startTestJob", withParameters:[:]) {
            (result: Any?, error: Error?) -> Void in
...
}

mtrezza commented 9 months ago

I think the old (5.4.0) parse-server API return the jobStatusId in the body. My app works well before I upgraded the server to parse-server 6.3.0.

I don't think so, this code hasn't changed in years:

https://github.com/parse-community/parse-server/blob/b70c2d9027c428bb6882337c5e56c4a7c3cabdf8/src/Routers/FunctionsRouter.js#L96-L101

In fact, it doesn't seem possible to return a custom body, the body is always {}. All job test cases expect the job ID in the header:

https://github.com/parse-community/parse-server/blob/b70c2d9027c428bb6882337c5e56c4a7c3cabdf8/spec/CloudCode.spec.js#L1966

You can verify this by installing Parse Server 5.4.0 - my guess is it will be the same behavior.

PFCloud.callFunction

Your code is calling a Cloud Function, not a Cloud Job. Maybe that's where the confusion comes form. If you need the job ID in the body, you could write a Cloud Function that calls the Cloud Job and then returns job ID in the response body.

Can we close this issue?

nebitrams commented 9 months ago

hi @mtrezza, I apologised that I was confused about iOS SDK. Indeed the iOS SDK doesn't matter.

I have further simplified my test case to focus on the return value of Parse.Cloud.startJob. I added some console.log. I compared the log between parse-server 5.4.0 vs 6.3.0. See the difference in the return value of Parse.Cloud.startJob.

parse-server 5.4.0 [startTestJob's console log result]

jobStatusId is YduU2JwgRm
YduU2JwgRm

parse-server 6.3.0 [startTestJob's console log result]

jobStatusId is [object Object]
{}

The cloud function that calls Parse.Cloud.startJob

Parse.Cloud.define("startTestJob", async (request) => {
  var jobId = await Parse.Cloud.startJob('testJob', request.params);
  console.log("jobStatusId is " + jobId);
  console.log(jobId);
  return jobId;
});

I see that there is no recent changes in the parse-server code. Is there any chance that environment parameter or node version matter in this case?

FYI, in running the parse-server 6.3.0, I added 3 parameters after I read the migration guide.

  allowClientClassCreation: false,
  allowExpiredAuthDataToken: false,
  encodeParseObjectInCloudFunction: true

mtrezza commented 9 months ago

What is the value of var jobId in your Cloud Function? What are you sending in request.params?

nebitrams commented 9 months ago

I did further test by varying parse-server version.

Parse.Cloud.startJob still returns jobStatusId in parse-server 5.5.0. However, starting from 6.0.0, it returns a dictionary (looks like empty).

nebitrams commented 9 months ago

What is the value of var jobId in your Cloud Function? What are you sending in request.params?

var jobId is a local variable that receive return value from await Parse.Cloud.startJob. The console.log prints the jobId's value for both test cases. See the console log for parse-server 5.4.0 and 6.3.0 in https://github.com/parse-community/parse-server/issues/8770

request.params is an empty dictionary {}. There is no change in behaviour when I omit request.params.

mtrezza commented 9 months ago

But what is the value of var jobId - an empty object?

I'd first compare the behavior of the REST API to the return value of the Parse JS SDK method. For both server versions. And post the request code and response values. It may be a change in the SDK, not in the server API.

nebitrams commented 9 months ago

jobId is an empty object. I used the following code to print jobId. Is there other way to inspect it?

jobId is [object Object]  // console.log("jobId is " + jobId);
{}  // console.log(jobId);

REST API has the JobStatusId in the response header, and empty body. Parse JS SDK output is as per what I wrote above. It returns an empty object.

mtrezza commented 9 months ago

REST API has the JobStatusId in the response header, and empty body. Parse JS SDK output is as per what I wrote above. It returns an empty object.

I assume that was for Parse Server 6 - could you try the same for Parse Server 5.5.0?

Since you already pinned the issue down to a specific Parse Sever version, I'd use that version, set breakpoints and inspect the code to understand how the status ID gets into the body.

nebitrams commented 9 months ago

This is the output I ran the same code against parse server 5.5.0

jobId is YduU2JwgRm  // console.log("jobId is " + jobId);
YduU2JwgRm  // console.log(jobId);

I observed different output between these 2 parse-server versions: 5.5.0 -> 6.0.0

mtrezza commented 9 months ago

Could you try the REST API of 5.5.0 and see what it returns?

I've quickly looked through the code of Parse Server 5.5.0 vs. 6.0.0 and JS SDK from both packages and couldn't find any difference. What you could do to analyze this issue is to use Parse Server 5.5.0 and set breakpoints, then run the job to investigate how the status ID gets into the body. From the REST API it seems that it should return an empty body, hence I suggested you test the REST API.

mtrezza commented 9 months ago

I've done some research and found:

The REST API returns the status ID only in the header
The JS SDK returns the status ID as a result of the Parse.Cloud.runJob, see: https://github.com/parse-community/Parse-SDK-JS/blob/e0b713e933772336e7dc61a0fc3fe40fe4ce5319/src/RESTController.js#L115-L117 where the RESTController copies the header value of x-parse-job-status-id to the response object. This makes sense, since in the Parse JS SDK there is no other place to return the status ID other than in the return value of Parse.Cloud.runJob.

I've transferred this issue to the Parse JS SDK repository since this behavior is specific to the Parse JS SDK.

There are also tests that require the status ID to be returned to pass, see: https://github.com/parse-community/Parse-SDK-JS/blob/e0b713e933772336e7dc61a0fc3fe40fe4ce5319/integration/test/ParseCloudTest.js#L96C1-L104

Since these tests are passing, it indicates that the status ID is returned. You could open a PR here, copy the test mentioned above and simplify it to just return the status ID and it should do that. If the test passes, I suggest you set breakpoints in a local Parse Server deployment in the RESTController code mentioned above to see whether the status ID is set, and if not, where it's lost, because the REST API of Parse Server 6 returns it, as you've already established.

nebitrams commented 9 months ago

@mtrezza, many thanks for identifying the module/code that might make Parse.Cloud.startJob not returning JobStatusId.

In your reply, you mentioned Parse.Cloud.runJob, but I am calling Parse.Cloud.startJob. Do you mean startJob instead?

I am not familiar with running unit test and debugging parse-server code using breakpoint. I usually just use console.log function to troubleshoot Cloud functions that I wrote. I will try to find some free time on weekend to learn these 2 skills.

Lastly, should this issue be tagged as type:bug instead of type:question?

dplewis commented 9 months ago

@nebitrams Can you try this again with directAccess: false server configuration? The issue is when directAccess is true (which was defaulted to true after 6.0.0) The RESTController in the JS SDK gets replaced so you don't have to use an http request since in Cloud Code the JS SDK and Server are on the same node runtime. The replacement Controller doesn't support JobId or PushId headers. I can do a PR with a fix.

nebitrams commented 9 months ago

I see. Your suspect make sense. I will try in approximately 6 hours time.

dplewis commented 9 months ago

@nebitrams I've made a PR if your interested. https://github.com/parse-community/parse-server/pull/8766

nebitrams commented 9 months ago

Awesome. Thanks for the PR.

Adding this parameter to parse-server 6.3.0 made the problem goes away!

Config:

directAccess: false

This is the output of Parse.Cloud.startJob, which works well.

jobId is EiyxdrresD
EiyxdrresD

parse-community / parse-server