incorrect textures

[monschine]

Hi,

I can download all tiles, also the model and textures and the rest of assets, but once the tour is loaded the textures looks wrong, like if there's some wrong uv values...

Not really sure, There is actually a loading process where the textures changes, so I'm not sure what's the issue. Do you success in viewing them correctly ? I have tested different tour ids DWuridgR5Fj

[mitchcapper]

Are you using the internal web server or how are you serving the files?

[Slyke]

This only happens when in "Dollhouse" or "Floorplan" mode. When in walk mode, everything appears fine.

Correct loading (from matterport.com):

When running locally:

[mitchcapper]

Are you using the internal web server or how are you serving the files?

and are you using the latest files from this repo?

[Slyke]

I'm using the webserver to run it: python3.10 matterport-dl.py XXXXXXXX 127.0.0.1 8080. The only thing I can think of is that I'm running this in WSL2 Ubuntu on Windows.

I git clone it a few hours back. The only changes I made to the files were adding 721 and 251 to the list of js files to download:

$ uname -a
Linux DESKTOP-8MFHJA9 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ git status
On branch main
Your branch is up to date with 'origin/main'.

$ git log
commit 9330314c0d9379b3d9885daf32f217ab5cb3a7b5 (HEAD -> main, origin/main, origin/HEAD)
Merge: b25ad72 e7683c6
Author: Jasper Rebane <rebane2001@gmail.com>
Date:   Fri Dec 17 05:42:07 2021 +0200

    Merge pull request #25 from mitchcapper/patch-1

    addiitonal new js files

$ git diff
diff --git a/matterport-dl.py b/matterport-dl.py
index 71eafff..c5cb0c1 100644
--- a/matterport-dl.py
+++ b/matterport-dl.py
@@ -134,7 +134,7 @@ def downloadGraphModels(pageid):

 def downloadAssets(base):
-    js_files = ["showcase","browser-check","30","79","134","136","164","250","321","356","423","464","524","539","606","614","666","764","828","833","947"]
+    js_files = ["showcase","browser-check","30","79","134","136","164","250","251","321","356","423","464","524","539","606","614","666", "721","764","828","833","947"]
     language_codes = ["af", "sq", "ar-SA", "ar-IQ", "ar-EG", "ar-LY", "ar-DZ", "ar-MA", "ar-TN", "ar-OM",
      "ar-YE", "ar-SY", "ar-JO", "ar-LB", "ar-KW", "ar-AE", "ar-BH", "ar-QA", "eu", "bg",
      "be", "ca", "zh-TW", "zh-CN", "zh-HK", "zh-SG", "hr", "cs", "da", "nl", "nl-BE", "en",

I do see some errors in the Javascript console:

http://127.0.0.1:8080/api/v1/jsonstore/model/trims/XXXXXXXX 404 (File not found)
http://127.0.0.1:8080/models/XXXXXXXXXXXXXXXXXX/assets/pan/low/~/XXXXXXXXXXXXXXXXXXXXXXXXXX_skybox1.jpg?t=2-XXXXXXXXXXXXXXXXXXXXX-1641265769-1&imgopt=1
[pin-mesh] 0.876s XXXXXXXXXXX failed to load texture: [object Object]

I can send you the matterport ID if you need to diagnose/test yourself.

[mitchcapper]

To confirm did you use the --advanced-download flag when downloading the matterport id? Without it it does not download the skybox and dollhouse textures. If you did not, you do not need to start from scratch just leave the existing folder in place, re-run the clone with --advanced-download and it should skip all the files you already downloaded.

[Slyke]

That was it! Did that fix it for you too @monschine ?

[monschine]

No it did not. could you try with this ID : DWuridgR5Fj run_report.log I just attached my log

Could you provide some ID which did work for you ? Thanks

[mitchcapper]

@monschine can you confirm you are using the internal web server for hosting? the run log seems like you did run with the advanced download flag.

[monschine]

Yes @mitchcapper I was trying to use the inetrnal web server to run it. But as you can see in the log there are many files that could not be downloaded, such as all the 4k tiles and also the model .dam which is not the 50k

[mitchcapper]

Well keep in mind we are a bit dumb in our fetching so many requests are expected to fail but it is easier than us trying to know exactly what files do and do not exist. Assuming you have downloaded a mass quantity that they started to block your IP im not sure the issue. I will try to run your ID this week.

[mitchcapper]

WFM, however I added some code to now detect any new js files so it is possible that broke it for you try PR #29 or wait for it to be merged in. I ran python matterport-dl.py https://my.matterport.com/show/?m=DWuridgR5Fj --advanced-download then python matterport-dl.py DWuridgR5Fj 127.0.0.1 20000 and it shows:

[monschine]

Hi again,

I can confirm that thanks to the extra js files, now the model can be run. Anyway I have taken a deeper look into the texture topic, because I was still experiencing issues. So if I do not use the 50k_texture_jpg_high (by renaming the folder) but just the low, it works fine. But if I would use the high quality there are always some missing tiles https://imgur.com/a/DdFhEfN

.jpgcrop=512,512,x0.25,y0.5 .jpgcrop=512,512,x0.5,y0.25 .jpgcrop=512,512,x0.5,y0.75 .jpgcrop=512,512,x0.75,y0.5

Those 4 tiles are always missing on my downloads, that makes me think is not an issue about request. I was proving that the issue is fixed by using the 2048*2048 image to crop the missing images.

I was taking a quick look at the code, but did not manage to find a quick fix, I was just thinking that we could only download the 2048*2048 image and tile it using PIL and numpy.

Could you confirm that when you downloaded DWuridgR5Fj with --advanced-download you get those missing file mentioned above ?

Thanks, and kind regards

[mitchcapper]

no plans to create our own images sounds like it may be a bit fragile that way. Right now the code for the advanced download tries to get the data from matterport and then has specific groups it tries I think the: ADV_CROP_FETCH of { "start":"crop=512,512,", "increment":'0.25' } would be what applies here.

it should start at 0 and go up by 0.25 for each param to get to 1.

I see it downloading: 3-0&crop=512,512,x0.25,y0.5 0&crop=512,512,x0.5,y0.75 for example which seems to match what you say are missing. That results in local files like _50k_000.jpgcrop=512,512,x0.25,y0.5 _50k_000.jpgcrop=512,512,x0.5,y0.75.jpg

Honestly start a clean folder, do a run. If you are missing a file look in the run_report.log. Did it request the file you said is missing? If so does it say it failed? If it did download successfully is it where it is expected ? i am guessing you are not seeing the local file on the hard drive.