Closed alexozwald closed 1 year ago
parth-io
commented
1 year ago Hi, I'm just wondering, if the file is unpublished by the instructor, how does it show up on your side? Because only instructors can view unpublished files. Is the module itself unpublished?
I guess we can try printing the FileResult and viewing the values of the locked or hidden fields.
https://canvas.instructure.com/doc/api/files.html
k-walter
commented
1 year ago @alexozwald Do you mean that
curl and share the json of that course using
curl 'https://northeastern.instructure.com/api/v1/courses' -H 'Authorization: Bearer <token>' | tee courses.json
alexozwald
commented
1 year ago @k-walter The course is published and I have access to it & some of the files in it. There are just some (inaccessible) files shown to me that I can't preview/download I believe because either the module or assignment is still hidden. I queried the curl statement you wrote but the "workflow_state" of every course was merely "available" as every course is published.
Per inference (2), yup I gotchu. This is all of the info from the failed api request for the file info.
{"errors":[{"message":"The specified resource does not exist."}]}{
"Request URL": "https://northeastern.instructure.com/api/lti/courses/null/jwt_token?tool_launch_url=https%3A%2F%2Fprod.ally.ac%2Fapi%2Fv1%2F9158%2Flti%2Finstitution",
"Request Method": "GET",
"Status Code": "404 ",
"Remote Address": "54.87.201.136:443",
"Referrer Policy": "no-referrer-when-downgrade"
}{
"cache-control": "no-cache",
"content-encoding": "br",
"content-security-policy": "frame-ancestors 'self' northeastern.instructure.com northeastern.staging.instructure.com northeastern.beta.instructure.com northeastern.test.instructure.com;",
"content-type": "application/json; charset=utf-8",
"date": "Wed, 11 Jan 2023 20:30:38 GMT",
"p3p": "CP="None, see http://www.instructure.com/privacy-policy"",
"referrer-policy": "no-referrer-when-downgrade",
"server": "Apache",
"set-cookie": "HIDDEN_FOR_YOU",
"set-cookie": "HIDDEN_FOR_YOU",
"status": "404 Not Found",
"strict-transport-security": "max-age=31536000",
"vary": "Accept-Encoding",
"x-a11y-ally": "Dana Danger Grey",
"x-canvas-meta": "q=9392;a=1;g=TlmmuJDJi06PmWbkhU736yDfZI3udCLXgBmneTIp;s=14523;c=cluster32;z=us-east-1e;o=external_tools;n=jwt_token;st=798f19ef5ab54576b56c68777fc4043a-06c4b3ad4db7b92e-0;b=1845576;m=1845576;u=0.02;y=0.00;d=0.00;",
"x-canvas-user-id": "145230000000048733",
"x-content-type-options": "nosniff",
"x-download-options": "noopen",
"x-permitted-cross-domain-policies": "none",
"x-rate-limit-remaining": "700.0",
"x-request-context-id": "d313cc2d-fe3e-4314-acc5-72a72f1be283",
"x-request-cost": "0.021374063998791826",
"x-request-processor": "0f033abdced739771",
"x-runtime": "0.033268",
"x-session-id": "b22ae95526d9ca1407731b9edac63d73",
"x-xss-protection": "1; mode=block"
}{
":authority": "northeastern.instructure.com",
":method": "GET",
":path": "/api/lti/courses/null/jwt_token?tool_launch_url=https%3A%2F%2Fprod.ally.ac%2Fapi%2Fv1%2F9158%2Flti%2Finstitution",
":scheme": "https",
"accept": "application/json, text/javascript, */*; q=0.01",
"accept-encoding": "gzip, deflate, br",
"accept-language": "en,es;q=0.9",
"cookie": "HIDDEN_FOR_YOU",
"dnt": "1",
"referer": "https://northeastern.instructure.com/files/folder/courses_133310/?preview=19173922",
"sec-ch-ua": ""Not?A_Brand";v="8", "Chromium";v="108", "Google Chrome";v="108"",
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": ""macOS"",
"sec-fetch-dest": "empty",
"sec-fetch-mode": "cors",
"sec-fetch-site": "same-origin",
"sentry-trace": "d40727b1e25844c598b581aaf12f72b5-9896ab6a064e7f90-0",
"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36",
"x-requested-with": "XMLHttpRequest"
}&& @parth-io I queried the results for the files API on locked & hidden like you suggested. I'm not sure if it would add a noticable amount of strain to the program, but, skiping if locked_for_user = true i think would solve our issue pretty simply. Although having encountered courses that have done this before: if somethings hidden but not locked I'd weigh in that it should still be downloaded.
$ curl 'https://northeastern.instructure.com/api/v1/courses/133310/files' -H 'Authorization: Bearer <TOKEN>' | tee courses.json[{
"id": 19173922,
"uuid": "MwcVlufsPQxFwZ9egAJibwQij4IQ6vpCig0Bnw9q",
"folder_id": 2092586,
"display_name": "1.47.pdf",
"filename": "1.47.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 1462486,
"created_at": "2022-09-14T17:13:20Z",
"updated_at": "2023-01-04T15:29:13Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-14T17:13:20Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19173922",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}, {
"id": 19173926,
"uuid": "aAb5esIocQJUmQN1N64ovacZfmv0DizdEySrlckh",
"folder_id": 2092586,
"display_name": "1.50.pdf",
"filename": "1.50.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 381586,
"created_at": "2022-09-14T18:38:22Z",
"updated_at": "2023-01-04T15:29:13Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-14T18:38:22Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19173926",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}, {
"id": 19173923,
"uuid": "yx3ZDus84303KzWpzBuUZMcj1CXjKLHPKCIrNN4r",
"folder_id": 2092586,
"display_name": "1.57.pdf",
"filename": "1.57.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 1628297,
"created_at": "2022-09-14T21:42:20Z",
"updated_at": "2023-01-04T15:29:13Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-14T21:42:20Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19173923",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}, {
"id": 19174014,
"uuid": "Mn84KJQ8Yg3ZXkM3eKX3bsZHoLhnwhd6D0T8bq48",
"folder_id": 2092586,
"display_name": "2.19.pdf",
"filename": "2.19.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 473865,
"created_at": "2022-09-20T15:15:09Z",
"updated_at": "2023-01-04T15:29:20Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-20T15:15:09Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19174014",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}, {
"id": 19174012,
"uuid": "U4RxR6tIzYxzfMjJvWfDbOS0L461etRUoJ58dz1S",
"folder_id": 2092586,
"display_name": "2.21.pdf",
"filename": "2.21.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 406040,
"created_at": "2022-09-16T01:38:57Z",
"updated_at": "2023-01-04T15:29:20Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-16T01:38:57Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19174012",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}, {
"id": 19174010,
"uuid": "PMmvFfUweufa4dETDchH8MtPG71YYXFp4rh1DSeX",
"folder_id": 2092586,
"display_name": "2.30.pdf",
"filename": "2.30.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 367195,
"created_at": "2022-09-16T01:38:55Z",
"updated_at": "2023-01-04T15:29:20Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-16T01:38:55Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19174010",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}, {
"id": 19174011,
"uuid": "Snxb0fpEJAsa6XlJbREGyja7TPsiV793cUjFk6p9",
"folder_id": 2092586,
"display_name": "2.45.pdf",
"filename": "2.45.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 628327,
"created_at": "2022-09-16T01:38:56Z",
"updated_at": "2023-01-04T15:29:20Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-16T01:38:56Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19174011",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}, {
"id": 19174009,
"uuid": "QyzFxnhRJSheXHVsdKrkVBG4NfKmrvCPUn1un94y",
"folder_id": 2092586,
"display_name": "2.47.pdf",
"filename": "2.47.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 1242827,
"created_at": "2022-09-16T01:38:53Z",
"updated_at": "2023-01-04T15:29:20Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-16T01:38:53Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19174009",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}, {
"id": 19174015,
"uuid": "86PfHjOAys6z8fovLUozohDTXbaoiD5rzzFDopLF",
"folder_id": 2092586,
"display_name": "2.61.pdf",
"filename": "2.61.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 371977,
"created_at": "2022-09-20T15:15:10Z",
"updated_at": "2023-01-04T15:29:20Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-20T15:15:10Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19174015",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}, {
"id": 19174016,
"uuid": "Au9U1d3ZlSBWNE4naQ7sHUGkQj7loPjHdfCyEZLd",
"folder_id": 2092586,
"display_name": "2.62.pdf",
"filename": "2.62.pdf",
"upload_status": "success",
"content-type": "application/pdf",
"url": "",
"size": 347814,
"created_at": "2022-09-20T15:15:12Z",
"updated_at": "2023-01-04T15:29:20Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": "",
"modified_at": "2022-09-20T15:15:12Z",
"mime_class": "pdf",
"media_entry_id": null,
"category": "uncategorized",
"locked_for_user": true,
"lock_info": {
"asset_string": "attachment_19174016",
"context_module": {
"id": 1031458,
"context_id": 133310,
"context_type": "Course",
"name": "Recommended Problems_Solutions",
"position": 3,
"prerequisites": [],
"completion_requirements": [],
"created_at": "2023-01-04T15:29:30Z",
"updated_at": "2023-01-04T15:29:31Z",
"workflow_state": "unpublished",
"deleted_at": null,
"unlock_at": null,
"migration_id": "gf021076c13e3e67bd6a346e61a62a753",
"require_sequential_progress": false,
"cloned_item_id": null,
"completion_events": null,
"requirement_count": null,
"root_account_id": 1
}
},
"lock_explanation": "This file is part of an unpublished module and is not available yet.",
"visibility_level": "inherit"
}] %@alexozwald I have the fix for it. Can you try it to see if it works for you? (ideally before the files are unlocked XD)
git clone -b fix-locked-files git@github.com:k-walter/canvas-downloader.git
cd canvas-downloader
cargo build
cd target/debug
./canvas-downloader -u https://northeastern.instructure.com -t MY_API_KEY -d dwnld/Response Headers "set-cookie": ... Request Headers "cookie": ...
Also, don't share your cookie in public
k-walter
commented
1 year ago Closing since there's been no complaints about this fix for a week.
Hey All,
By alphabetical order, i'm pretty sure the error occurred on a course that I just started, and that the error came from trying to download an "unpublished" file from a class files repo. If you try to preview it online you get the following msg: "This file is part of an unpublished module and is not available yet." but if you hit "...">"Download" online it just refreshes the page lol.
I would happily fix the code myself but unfortunately I have no Rust experience to help y'all :'). If you have any thoughts or I can help in any other way though do let me know!
debug info: