Download exception favicon.ico not found

[icantcode333]

Hi, thanks for this tool, not working for me though using mu-ramadan's version (rebane version failed with different errors). See error during download:

C:\Users\Kate\AppData\Local\Programs\Python\Python312>py C:\Users\Kate\AppData\Local\Programs\Matterport2\matterport-dl-main\matterport-dl.py "https://my.matterport.com/show/?m=vsFKMhVjwcf" --advanced-download Downloading base page... Doing advanced download of dollhouse/floorplan data... Downloading static assets... Traceback (most recent call last): File "C:\Users\Kate\AppData\Local\Programs\Matterport2\matterport-dl-main\matterport-dl.py", line 129, in downloadFile response.raise_for_status() # Raise an exception if the response has an error status code ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Kate\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://my.matterport.com/favicon.ico

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\Kate\AppData\Local\Programs\Matterport2\matterport-dl-main\matterport-dl.py", line 689, in initiateDownload(pageId) File "C:\Users\Kate\AppData\Local\Programs\Matterport2\matterport-dl-main\matterport-dl.py", line 554, in initiateDownload downloadPage(getPageId(url)) File "C:\Users\Kate\AppData\Local\Programs\Matterport2\matterport-dl-main\matterport-dl.py", line 533, in downloadPage downloadAssets(staticbase) File "C:\Users\Kate\AppData\Local\Programs\Matterport2\matterport-dl-main\matterport-dl.py", line 208, in downloadAssets downloadFile("https://my.matterport.com/favicon.ico", "favicon.ico") File "C:\Users\Kate\AppData\Local\Programs\Matterport2\matterport-dl-main\matterport-dl.py", line 154, in downloadFile raise Exception Exception

[mitchcapper]

Try https://github.com/rebane2001/matterport-dl/pull/111 im still revising a few things but we should also have it merged into master shortly if you prefer to wait.

[icantcode333]

Hi thank you for you help. I made a copy of my Matterport2 folder (mu-ramadan's download) and then overwrote that with the updated files in #111. I had to run an additional python install for curl_cffi and then when I try the download I have a new error. Have I missed a step?

C:\Users\Kate\AppData\Local\Programs\Python\Python312>py C:\Users\Kate\AppData\Local\Programs\Matterport3\matterport-dl-main\matterport-dl.py "https://my.matterport.com/show/?m=vsFKMhVjwcf" --advanced-download Downloading base page... https://my.matterport.com/show/?m=vsFKMhVjwcf Doing advanced download of dollhouse/floorplan data... Downloading static assets... Downloading model info... Downloading images... Downloading graph model data... Patching graph_GetModelDetails.json URLs Traceback (most recent call last): File "C:\Users\Kate\AppData\Local\Programs\Matterport3\matterport-dl-main\matterport-dl.py", line 740, in initiateDownload(pageId) File "C:\Users\Kate\AppData\Local\Programs\Matterport3\matterport-dl-main\matterport-dl.py", line 588, in initiateDownload downloadPage(getPageId(url)) File "C:\Users\Kate\AppData\Local\Programs\Matterport3\matterport-dl-main\matterport-dl.py", line 580, in downloadPage patchGetModelDetails() File "C:\Users\Kate\AppData\Local\Programs\Matterport3\matterport-dl-main\matterport-dl.py", line 372, in patchGetModelDetails with open(f"api/mp/models/graph_GetModelDetails.json", "r", encoding="UTF-8") as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'api/mp/models/graph_GetModelDetails.json'

[mitchcapper]

hrm that is odd running the files from my PR it does seem to be working for me:

try going to: https://github.com/mitchcapper/matterport-dl/tree/update_wrapup hit the arrow net to the code word in the green button and click download zip. Extract that to a fresh folder and try running your command again.

[icantcode333]

downloaded your zip to a new folder and running again seems to have the same error:

C:\Users\Kate\AppData\Local\Programs\Python\Python312>py C:\Users\Kate\AppData\Local\Programs\matterport_mitch\matterport-dl.py "https://my.matterport.com/show/?m=vsFKMhVjwcf" --advanced-download Downloading base page... https://my.matterport.com/show/?m=vsFKMhVjwcf Doing advanced download of dollhouse/floorplan data... Downloading static assets... Downloading model info... Downloading images... Downloading graph model data... Patching graph_GetModelDetails.json URLs Traceback (most recent call last): File "C:\Users\Kate\AppData\Local\Programs\matterport_mitch\matterport-dl.py", line 740, in initiateDownload(pageId) File "C:\Users\Kate\AppData\Local\Programs\matterport_mitch\matterport-dl.py", line 588, in initiateDownload downloadPage(getPageId(url)) File "C:\Users\Kate\AppData\Local\Programs\matterport_mitch\matterport-dl.py", line 580, in downloadPage patchGetModelDetails() File "C:\Users\Kate\AppData\Local\Programs\matterport_mitch\matterport-dl.py", line 372, in patchGetModelDetails with open(f"api/mp/models/graph_GetModelDetails.json", "r", encoding="UTF-8") as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'api/mp/models/graph_GetModelDetails.json'

[mitchcapper]

hrm odd your versions seem mostly up to date but lets verify:

PS s:\Sandbox\ktest\matterport-dl-update_wrapup> wget https://codeload.github.com/mitchcapper/matterport-dl/zip/refs/heads/update_wrapup -O out.zip PS s:\Sandbox\ktest\matterport-dl-update_wrapup> unzip .\out.zip PS s:\Sandbox\ktest\matterport-dl-update_wrapup> python matterport-dl.py "https://my.matterport.com/show/?m=vsFKMhVjwcf" --advanced-download

Downloading base page... https://my.matterport.com/show/?m=vsFKMhVjwcf
Doing advanced download of dollhouse/floorplan data...
Downloading static assets...
Downloading model info...
Downloading images...
Downloading graph model data...
Patching graph_GetModelDetails.json URLs
Downloading model ID: vsFKMhVjwcf ...

PS s:\Sandbox\ktest\matterport-dl-update_wrapup> python -V Python 3.12.4

PS s:\Sandbox\ktest\matterport-dl-update_wrapup> pip list

Package            Version
------------------ --------
certifi            2024.6.2
cffi               1.16.0
charset-normalizer 3.3.2
colorama           0.4.6
curl_cffi          0.6.4
idna               3.7
iso8601            2.1.0
pip                24.0
requests           2.32.3
tqdm               4.66.4
urllib3            2.2.2

[mitchcapper]

Also in the look at the .\vsFKMhVjwcf\run_report.log it should have something like:

2024-07-03 16:47:20 DEBUG    Starting REQ for GRAPH_MODEL 380: should exist: True  File: api/mp/models/graph_GetModelDetails.json at url: https://my.matterport.com/api/mp/models/graph GetModelDetails
2024-07-03 16:47:20 INFO     Finished REQ for GRAPH_MODEL 380: should exist: True  File: api/mp/models/graph_GetModelDetails.json at url: https://my.matterport.com/api/mp/models/graph GetModelDetails

if not send the end of the log file might give some hints.

[icantcode333]

Looks like I have the same versions except I am missing iso8601

C:\Users\Kate\AppData\Local\Programs\Python\Python312>python -V Python 3.12.4

C:\Users\Kate\AppData\Local\Programs\Python\Python312>cd scripts

C:\Users\Kate\AppData\Local\Programs\Python\Python312\Scripts>pip list Package Version

---

certifi 2024.6.2 cffi 1.16.0 charset-normalizer 3.3.2 colorama 0.4.6 curl_cffi 0.7.0 idna 3.7 pip 24.0 pycparser 2.22 requests 2.32.3 tqdm 4.66.4 urllib3 2.2.2

[notice] A new release of pip is available: 24.0 -> 24.1.1 [notice] To update, run: C:\Users\Kate\AppData\Local\Programs\Python\Python312\python.exe -m pip install --upgrade pip

I'll try to attach the log file, there are many errors reported. run_report.log

Thanks again, really appreciate your help :-)

[icantcode333]

So good news and bad news. I tried the process on a Mac and was able to download the model successfully with no errors:

jeff@Jeffs-Air matterport % python3 matterport-dl.py "https://my.matterport.com/show/?m=vsFKMhVjwcf" --advanced-download Downloading base page... https://my.matterport.com/show/?m=vsFKMhVjwcf Doing advanced download of dollhouse/floorplan data... Downloading static assets... Downloading model info... Downloading images... Downloading graph model data... Patching graph_GetModelDetails.json URLs Downloading model ID: vsFKMhVjwcf ... 100%|█████████████████████████████████████| 16830/16830 [04:55<00:00, 56.97it/s] Done!

However when I try to load the page in a browser it displays the static image of the house with the loading circle but it doesn't load the model.

Server log shows:

2024-07-04 11:56:23 INFO Server started up 2024-07-04 11:56:34 INFO Redirecting /js/showcase.js => /js/showcase-internal.js as using our internal showcase.js file 2024-07-04 11:56:34 INFO Handling a post request on /api/mp/models/graph?operation=GetModelPolicies: graph of operationName: GetModelPolicies we are handling internally 2024-07-04 11:56:34 INFO Handling a post request on /api/mp/models/graph?operation=GetLayers: graph of operationName: GetLayers we are handling internally 2024-07-04 11:56:34 INFO Handling a post request on /api/mp/models/graph?operation=GetModelViewPrefetch: graph of operationName: GetModelViewPrefetch we are handling internally 2024-07-04 11:56:34 INFO Handling a post request on /api/mp/models/graph?operation=GetModelDetails: graph of operationName: GetModelDetails we are handling internally 2024-07-04 11:56:34 INFO Handling a post request on /api/mp/models/graph?operation=GetSnapshots: graph of operationName: GetSnapshots we are handling internally 2024-07-04 11:56:34 WARNING 404 error: /models/501c1de4b586426c9de2ab75d69a4e85/assets/mesh_tiles/~/3.json?t=2-89cfd20f88a75cba72a25d7f991e4b2d67c4fa84-1720060280-1&k=models%2F501c1de4b586426c9de2ab75d69a4e85%2Fassets%2Fmesh_tiles may not be downloading everything right

There is no mesh_tiles subfolder in the /models/501c1de4b586426c9de2ab75d69a4e85/assets/ folder, so something is not quite right

[mitchcapper]

Oh. I think I see the error. I was silly, the graph model items it reads from the local repo for doing the fetch requests (graph_posts). In yours it doesn't attempt to download any. I am guessing on windows you are NOT in the matterport-dl directory when you are doing this. That would result in the relative path being wrong and it skipping them. infact looking above I see that: C:\Users\Kate\AppData\Local\Programs\Python\Python312>py C:\Users\Kate\AppData\Local\Programs\matterport_mitch\matterport-dl.py

You are running it from the python dir. try "cd C:\Users\Kate\AppData\Local\Programs\matterport_mitch\" then run C:\Users\Kate\AppData\Local\Programs\Python\Python312\py .\matterport-dl.py

I will fix the relative path issue (aka make window work).

As for mac or windows try the following for a more complete model download: --no-tilda --advanced-download --brute-js

The no-tilda in particular is likely key for mac. ~ by default represents the users home directory, but matterport also uses it in a folder path.

I only test on windows (no mac) but in theory it should work with that option.

[icantcode333]

Yes you were right, the path issue was the reason the download was failing on Windows. The following command worked (although the "py" abbreviation was not recognised, I used "python" instead):

C:\Users\Kate\AppData\Local\Programs\matterport_mitch>C:\Users\Kate\AppData\Local\Programs\Python\Python312\python .\matterport-dl.py "https://my.matterport.com/show/?m=vsFKMhVjwcf" --advanced-download Downloading base page... https://my.matterport.com/show/?m=vsFKMhVjwcf Doing advanced download of dollhouse/floorplan data... Downloading static assets... Downloading model info... Downloading images... Downloading graph model data... Patching graph_GetModelDetails.json URLs Downloading model ID: vsFKMhVjwcf ... 100%|█████████████████████████████████████████████| 16830/16830 [06:33<00:00, 42.82it/s] Done!

However I get the same problem as on the Mac trying to view: the page loads locally but is stuck on the Loading circle which fails to load the model. The command was:

C:\Users\Kate\AppData\Local\Programs\matterport_mitch>C:\Users\Kate\AppData\Local\Programs\Python\Python312\python .\matterport-dl.py "https://my.matterport.com/show/?m=vsFKMhVjwcf" 127.0.0.1 8080 View in browser: http://127.0.0.1:8080

The server.log shows:

2024-07-04 16:00:49 INFO Server started up 2024-07-04 16:01:01 INFO Redirecting /js/showcase.js => /js/showcase-internal.js as using our internal showcase.js file 2024-07-04 16:01:01 INFO Redirecting /js/showcase.js => /js/showcase-internal.js as using our internal showcase.js file 2024-07-04 16:01:02 INFO Handling a post request on /api/mp/models/graph?operation=GetModelPolicies: graph of operationName: GetModelPolicies we are handling internally 2024-07-04 16:01:02 INFO Handling a post request on /api/mp/models/graph?operation=GetLayers: graph of operationName: GetLayers we are handling internally 2024-07-04 16:01:02 INFO Handling a post request on /api/mp/models/graph?operation=GetModelViewPrefetch: graph of operationName: GetModelViewPrefetch we are handling internally 2024-07-04 16:01:02 INFO Handling a post request on /api/mp/models/graph?operation=GetModelDetails: graph of operationName: GetModelDetails we are handling internally 2024-07-04 16:01:02 INFO Handling a post request on /api/mp/models/graph?operation=GetSnapshots: graph of operationName: GetSnapshots we are handling internally 2024-07-04 16:01:02 WARNING 404 error: /models/501c1de4b586426c9de2ab75d69a4e85/assets/mesh_tiles/~/3.json?t=2-97285b41e2fd0974c52bedf344118f2b4ce816bf-1720075993-1&k=models%2F501c1de4b586426c9de2ab75d69a4e85%2Fassets%2Fmesh_tiles may not be downloading everything right

And the end of the cmd window logging when I Ctrl+C to end it is:

127.0.0.1 - - [04/Jul/2024 16:01:02] code 404, message File not found 127.0.0.1 - - [04/Jul/2024 16:01:02] "GET /models/501c1de4b586426c9de2ab75d69a4e85/assets/mesh_tiles/~/3.json?t=2-97285b41e2fd0974c52bedf344118f2b4ce816bf-1720075993-1&k=models%2F501c1de4b586426c9de2ab75d69a4e85%2Fassets%2Fmesh_tiles HTTP/1.1" 404 - 127.0.0.1 - - [04/Jul/2024 16:01:02] "GET /api/v1/jsonstore/model/trims/vsFKMhVjwcf HTTP/1.1" 200 - 127.0.0.1 - - [04/Jul/2024 16:01:02] "GET /fonts/mp-font.woff2?d6537ec55d06e0dbba9085b9bf2bcc09 HTTP/1.1" 200 - 127.0.0.1 - - [04/Jul/2024 16:01:02] "GET /fonts/roboto-700.woff2 HTTP/1.1" 200 - 127.0.0.1 - - [04/Jul/2024 16:01:12] "GET /images/nav_help_mouse_ctrl_click.png HTTP/1.1" 200 - 127.0.0.1 - - [04/Jul/2024 16:01:12] "GET /images/nav_help_mouse_drag_right.png HTTP/1.1" 200 - 127.0.0.1 - - [04/Jul/2024 16:02:02] "POST /api/v1/event HTTP/1.1" 200 - Traceback (most recent call last): File "C:\Users\Kate\AppData\Local\Programs\matterport_mitch\matterport-dl.py", line 750, in httpd.serve_forever() File "C:\Users\Kate\AppData\Local\Programs\Python\Python312\Lib\socketserver.py", line 235, in serve_forever ready = selector.select(pollinterval) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Kate\AppData\Local\Programs\Python\Python312\Lib\selectors.py", line 323, in select r, w, = self._select(self._readers, self._writers, [], timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Kate\AppData\Local\Programs\Python\Python312\Lib\selectors.py", line 314, in _select r, w, x = select.select(r, w, w, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ KeyboardInterrupt ^C

Should there be a mesh_tiles folder within the assets folder? I only have the .dam file and a ~ folder: Directory of C:\Users\Kate\AppData\Local\Programs\matterport_mitch\vsFKMhVjwcf\models\501c1de4b586426c9de2ab75d69a4e85\assets

04/07/2024 03:53 PM

. 04/07/2024 03:53 PM .. 04/07/2024 03:53 PM 1,456,922 c2b10e2a442c47228651773288b55234_50k.dam 04/07/2024 03:53 PM ~ 1 File(s) 1,456,922 bytes 3 Dir(s) 161,557,307,392 bytes free

On the Mac, the additional parameters meant the downloaded folder was bigger in file size, but still no mesh_tiles subfolder and have the same problem with trying to view the model.

[mitchcapper]

redownload my branch some large changes to support newer style modeling. Needs much more polish but working for showing me your model now. Make sure if using --no-tilda you do it for both the initial run and when doing the display (ie python .\matterport-dl.py "https://my.matterport.com/show/?m=vsFKMhVjwcf" 127.0.0.1 8080 --no-tilda)

[icantcode333]

Mitch you're a rockstar!! Working now on Mac (haven't tried PC). Thanks so much for your help :-)

[JosueAChavez]

Thank you so much! This works for me perfectly on WSL using venv. I did not have to include the --no-tilde to get it viewable locally.

[mitchcapper]

thanks, handled!