astuder / lightroom-map-fix

Fixing the Map module in Lightroom Classic
124 stars 17 forks source link

Map page is blank, was working earlier #41

Open rgry opened 1 year ago

rgry commented 1 year ago

First, Thank you for a great hack! Even though I think Adobe should have fixed it in the first place....

I had you fix running on my mac for over a year now, but not it does not work any more? Just for testing I installed LR6 on my windows and gone through the steps and it does not work here either? Map page is kind of blank? Not showing any errors?

I'm I the only one? or did Adobe/Google change something? Where do I look for a log?

Peter-MI commented 1 year ago

Same here on Windows 11. Google first showed error in the account console and Lightroom showed “request denied” but now it just doesn’t work anymore

astuder commented 1 year ago

Uh, oh! SaaS bit rot may finally catch up with us. 😢

I also get the blank screen. But didn't get a "request denied" message and don't see errors in the Google cloud account console. The only thing I see in the cloud console, is that there are calls to the Geocoding API but no calls to the Maps Javascript API. I see both calls on Feb19 when I last used Lightroom maps.

If anyone has more details about error messages, or maybe even a HTTP logs, please share. But don't forget to redact API keys.

astuder commented 1 year ago

I wonder if this is related to deprecation of Google Maps JavaScript API V2 https://developers.google.com/maps/documentation/javascript/v2tov3

Peter-MI commented 1 year ago

But they say V2 was made unavailable in May 2021… also the OSM hack does not work anymore: bottom left menu does only show LR original map styles

astuder commented 1 year ago

I looked at this today, and can't figure out where it fails. It seems that it fails creating the map view even before calling Google.

Can everyone that encounters the issue please state their platform? I'm on Windows 11 like Peter. Any Mac users?

senarvi commented 1 year ago

Same thing with me on Windows 11.

teskeyn commented 1 year ago

I'm still on WIndows 10 and have the same symptoms with LR6. I noticed in my Google cloud console that there was a recommendation that I restrict my key to adobe.com, which I don't recall seeing before. Is LR6 actually sending the request to Google via adobe.com, and Adobe have now blocked it; but that doesn't make any sense to me and wouldn't explain why the OSM hack fails

softnorbi commented 1 year ago

Exactly the same on my PC with Windows 10 😢 Only the reverse geocoding ("LR adresss suggestions") still works for the GPS data in the picture. Map search however e.g. for the state name doesn't work as well. I just looked on the google maps platform and my last recognized requests to MapsJavaScript API have been on Thursday 23. Feb. , without any errors recordet at that time and since then...

astuder commented 1 year ago

Any Mac users? Never mind, just noticed that the original poster is running on Mac. So definitely not a platform issue.

@teskeyn do you have more details about the recommendation regarding key restriction? A newly enforced security measure (like CORS) sounds like a "good" candidate.

teskeyn commented 1 year ago

The only thing I did with the key restrictions was to remove all restrictions and LR6 map still didn't load. For now I've decided to add another step to my workflow using the new beta version of Geosetter https://geosetter.de/geosetter_beta.exe

https://geosetter.de/geosetter_beta.exe https://geosetter.de/geosetter_beta.exe

On Mon, 27 Feb 2023 at 18:58, Adrian Studer @.***> wrote:

Any Mac users?

@teskeyn https://github.com/teskeyn do you have more details about the recommendation regarding key restriction? A newly enforced security measure (like CORS) sounds like a "good" candidate.

— Reply to this email directly, view it on GitHub https://github.com/astuder/lightroom-map-fix/issues/41#issuecomment-1446884840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH4XGR7RFEDZNB3N7DD75ELWZT2MBANCNFSM6AAAAAAVH352GU . You are receiving this because you were mentioned.Message ID: @.***>

Peter-MI commented 1 year ago

@teskeyn: interestingly the old Geosetter now also does not work anymore - map requests end with "01.03.2023 15:51:12 ERROR: Could not complete the operation due to error 80020101 showMap3(2, 4, 45.4741962777, 9.1738253847, 10);"

teskeyn commented 1 year ago

Yes, but the new 4.2 beta does work.

On Wed, 1 Mar 2023, 14:57 Peter-MI, @.***> wrote:

@teskeyn https://github.com/teskeyn: interestingly the old Geosetter now also does not work anymore - map requests end with "01.03.2023 15:51:12 ERROR: Could not complete the operation due to error 80020101 showMap3(2, 4, 45.4741962777, 9.1738253847, 10);"

— Reply to this email directly, view it on GitHub https://github.com/astuder/lightroom-map-fix/issues/41#issuecomment-1450286115, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH4XGR6YMXQ26RQXSMFVNI3WZ5PUHANCNFSM6AAAAAAVH352GU . You are receiving this because you were mentioned.Message ID: @.***>

astuder commented 1 year ago

Oh, that's interesting about Geosetter. The two relevant tickets seem to be: https://geosetter.de/mantis/view.php?id=2261 https://geosetter.de/mantis/view.php?id=2256 A quick search in location.lrmodule didn't bring up showMap3, but maybe there are similarities with our issue.

Unfortunately, I won't have time the next few days to dig deeper into this.

diabWH3 commented 1 year ago

Just chiming in to say that I'm also experiencing this issue with Windows 10 and Lightroom Classic 6.13. This fix has allowed me to keep using this paid-for software for far longer than Adobe would have otherwise allowed, so I appreciate all the hard work of this community!

diabWH3 commented 1 year ago

This may be related, but I no longer seem to be able to manually adjust the Country/State/City/Sublocation metadata fields for my photos. In the Grid/Loupe views, I can manually type something into those metadata fields, but upon hitting Enter, the application does not save my entries.

DISREGARD: I restarted Lightroom and the Metadata panel works as intended. Just a weird glitch.

cyr06 commented 1 year ago

Arrghhhh this fix was so nice, I will stay tuned if someone find another workaround.

As we are all trying to make LR works the way we bought it, for Mac users I managed to make face recognition works even in march 23 ;)

diabWH3 commented 1 year ago

Thanks for doing that! I found your fix not looking after you posted it and was SO relieved to be able to limp the program along again, even without the automated facial recognition. The face tagging feature is a very important part of my workflow and it's great to be able to do that still, even if manually.

On Sun, Mar 12, 2023, 1:10 AM cyr06 @.***> wrote:

Arrghhhh this fix was so nice, I will stay tuned if someone find another workaround.

As we are all trying to make LR works the way we bought it, for Mac users I managed to make face recognition works even in march 23 ;) https://www.reddit.com/r/Lightroom/comments/knjkk0/lightroom_6xx_facial_recognition_causing_app/

— Reply to this email directly, view it on GitHub https://github.com/astuder/lightroom-map-fix/issues/41#issuecomment-1465133631, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL2VMAO3L73RVIIOCMNRDUDW3WHGZANCNFSM6AAAAAAVH352GU . You are receiving this because you commented.Message ID: @.***>

astuder commented 1 year ago

No real progress so far, but a short update of what I've tried so far:

I used unluac to decompile the Lua files. While the decompiler works, unfortunately the output is hard to read. I also wasn't able to recompile. Based on the Lua source, there seems to be some kind of debug console that can be enabled. No success to do that so far.

I dug into Lightroom and associated dll's using Ghidra. There are a few useful symbols which helps finding areas of interest. The Lua engine is implemented in AgKernel.dll and initialized in wichitafoundation.dll. Tried to set the value of the DEBUG Lua variable to True by patching wichitafoundation.dll, unfortunately without any effect.

I patched the window.onerror functions in LOCATIONMAPVIEW.LUA to display a pop-up with alert(). This only shows a message box when pressing the CTRL key, but nothing while launching the map module. I think this indicates that the map view window gets initialized, but still no clue where and why it fails.

astuder commented 1 year ago

DANGER ZONE !!!

For those playing along at home, this patch file will display the JavaScript code behind the map view. This patch breaks the map module, so make triple-sure that you have a backup!

# Dump styles and javascript in map view, this will break the map module!!
# patchluastr.py LOCATIONMAPVIEW.LUA -p dumpscript.patch -o LOCATIONMAPVIEW-dumpscript.bin
< <script
> <pre
< </script
> </pre
< <style
> <pre
< </style
> </pre

image

Anyone with experience integrating GoogleMaps seeing an obvious issue? The Google Maps URL does return the JS, though that even works with an invalid key

I see that the URL includes Adobe's signature, which we had to patch out for reverse-geocoding to work. Don't know where that's coming from or whether it's relevant. (nevermind, I forgot to apply the nature->street patch)

Any ideas on how to copy & paste that code? Currently hotkeys and right-click are captured by Lightroom.

astuder commented 1 year ago

Also, what's up with http://app-storage:51000? Is that a general browser/JS feature or specific to Lightroom?

pbb72 commented 1 year ago

Interesting! I haven't been using the map module in a while now, but I'll see what I can figure out.

From reading the error descriptions, it sounds like we'll have to rewrite the complete Javascript map code in Lightroom, updating it to the current version of Google Maps, AND write a layer around it to make Lightroom think we're still in the old Google Maps. So, not trivial, but I haven't looked yet, so cross your fingers.

astuder commented 1 year ago

I just confirmed with alert boxes that GoogleMaps invokes initialize (twice), and that it runs to completion without throwing exceptions.

senarvi commented 1 year ago

Any ideas on how to copy & paste that code? Currently hotkeys and right-click are captured by Lightroom.

The Microsoft OCR engine doesn't seem to work at all for source code. Google Drive works better, but you still have to fix the line breaks in some places. I saved the image to Google Drive and right-click > Open with > Google Docs.

Edite: deleted the source code, just to be sure to avoid any copyright concerns.
pbb72 commented 1 year ago

@astuder I haven't tried this, but what if you use <textarea> instead of <pre>?

Tibashoult commented 1 year ago

Hi, I have same issue with LR6.14 / Windows 10. My API is not restricted and no error showed on Google panel. Also, when I go to map section in LR, the last position name (city) I used one month ago i click is showed on top right corner. if this information can be usefull, and each time I try to show picture, I have one more connection in my google statistic panel. That's mean LR use well the API, but not show the map. Hope the community find a solution.

astuder commented 1 year ago

@astuder I haven't tried this, but what if you use <textarea> instead of \<pre\>?

I managed to mark the text with the mouse and drag & drop it into Notepad. Now thinking about how I can get Lightroom to output the full HTML :) ...maybe CDATA?

astuder commented 1 year ago

This works to get full source of the map view (minus HTML tags at start and end):

< <html>
> <html><textarea><![CDATA[
< </html>
> ]]></textarea></html>

As mentioned above, the text can be copied by marking it with the mouse and drag & dropping it into a text editor.

Hopefully this helps with tracking down where things break.

astuder commented 1 year ago

Well, that sucks.. the view as standalone HTML page works (after editing app-storage URL to point to Google Maps) :-/

Screenshot 2023-03-14 114042

astuder commented 1 year ago

After extracting the missing JS libraries with ResourceHacker, there's only one error not related to failed calls to the Lightroom backend:

Uncaught TypeError: projection is null
    getExtendedBounds file:///C:/Users/adrian/OneDrive/Code/Projects/lightroom-map-fix/mapview/markerclusterer.js:570
    MarkerClusterer file:///C:/Users/adrian/OneDrive/Code/Projects/lightroom-map-fix/mapview/markerclusterer.js:217

Here's the failing code:

/**
 * Extends a bounds object by the grid size.
 *
 * @param {google.maps.LatLngBounds} bounds The bounds to extend.
 * @return {google.maps.LatLngBounds} The extended bounds.
 */
MarkerClusterer.prototype.getExtendedBounds = function(bounds) {
[..]
  var projection = this.getProjection();
[..]
  // Convert the points to pixels and the extend out by the grid size.
  var trPix = projection.fromLatLngToDivPixel(tr);

I'm not familiar with Google Maps, but I think this.getProjection() is an API call to Google's code.

astuder commented 1 year ago

Ok, that was a red herring.. after fixing the escaping of the version number requested from Google Maps (v3.12) the error disappears. I now get a warning:

Google Maps JavaScript API warning: RetiredVersion https://developers.google.com/maps/documentation/javascript/error-messages#retired-version

But, the same warning is shown when using the version mentioned in the GeoSetter threads (v3.47). So I don't think that's the issue.

Next step probably requires digging into the Lua code.

cyr06 commented 1 year ago

Breaking news : it works again, I think the only thing I did was to connect to my google console account (I never do that) and make another billing account. Even if I don't know what it is :)

diabWH3 commented 1 year ago

You're able to see map tiles again? Could you outline the specific steps you took within the Google Cloud platform?

Message ID: @.***>

cyr06 commented 1 year ago

You have to know that I'm very bad at google's thing and I had no idea of what I meant to do. So connection to console.google I saw that my billing account was with a red flag. Maybe I click on manage and then renew it (maybe or something like that I don't have this menu any more). I remember that the red flag become orange or green (not red for sure). If I knew it was going to work I would have make some screenshot.

cyr06 commented 1 year ago

Maybe something : I only used the first part of @astuder workaround (it was in 2019 I think ;)

Tibashoult commented 1 year ago

Always not work for me. My google console is green. No problem. Payment card active. Very strange ! Hope somebody find a solution for all.

Tibashoult commented 1 year ago

Hi I try to delete my current credit card, by another. no map I try to desactivate the 2 API. Error message in LR with no found google account... And re-activate them. no map.

Perhaps we need te generated a new google key ? someone has tested this way ?

senarvi commented 1 year ago

You have to know that I'm very bad at google's thing and I had no idea of what I meant to do.

@cyr06 if you go to https://console.cloud.google.com/logs , you might see what changes you made to your account.

senarvi commented 1 year ago

I tried enabling and disabling various stuff in the billing account and the project in Google Cloud console. Two times I happened to get a "Map error: request denied" in Lightroom and I got two errors in the Geocoding API error count in the console. The error count is also increased if I search for a place name that the API doesn't find. If I search for a valid place name, there's no error and Lightroom confirms the address. The request count in the console is increased.

However, the request count in the Maps JavaScript API is never increased by Lightroom. I'm able to access both APIs using the API key via https://maps.googleapis.com/maps/api/js?key=...&callback=... and https://maps.googleapis.com/maps/api/geocode/json?address=...&key=.... Both geocoding and maps work and the request counts are increased this way.

cyr06 commented 1 year ago

You have to know that I'm very bad at google's thing and I had no idea of what I meant to do.

@cyr06 if you go to https://console.cloud.google.com/logs , you might see what changes you made to your account.

my log is empty ... :(

senarvi commented 1 year ago

my log is empty ... :(

Did you notice that it only shows last hour or so? You can click Edit time and select for example Last 7 days.

cyr06 commented 1 year ago

I noticed but it was before I thought ;)

{
  "protoPayload": {
    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
    "authenticationInfo": {
      "principalEmail": « bbbb@jjjjj »
    },
    "requestMetadata": {
      "callerIp": « xxxxxxxxxxxx »,
      "requestAttributes": {},
      "destinationAttributes": {}
    },
    "serviceName": "cloudbilling.googleapis.com",
    "methodName": "AssignResourceToBillingAccount",
    "authorizationInfo": [
      {
        "resource": "billingAccounts/xxxxxxxxxx »,
        "permission": "billing.resourceAssociations.create",
        "granted": true,
        "resourceAttributes": {}
      },
      {
        "resource": "projects/xxxxx-xxxxx »x,
        "permission": "resourcemanager.projects.createBillingAssignment",
        "granted": true,
        "resourceAttributes": {}
      },
      {
        "resource": "billingAccounts/xxxxxxx »,
        "permission": "billing.resourceAssociations.create",
        "granted": true,
        "resourceAttributes": {}
      },
      {
        "resource": "projects/xxxxx »xx,
        "permission": "resourcemanager.projects.deleteBillingAssignment",
        "granted": true,
        "resourceAttributes": {}
      },
      {
        "resource": "billingAccounts/xxxxxx »,
        "permission": "billing.resourceAssociations.delete",
        "granted": true,
        "resourceAttributes": {}
      }
    ],
    "resourceName": "projects/lxxxxx »,
    "request": {
      "billingAccountName": "billingAccounts/xxxxxx »,
      "resourceName": "projects/xxxxxx »,
      "@type": "type.googleapis.com/google.internal.cloudbilling.billingaccount.v1.AssignResourceToBillingAccountRequest"
    }
  },
  "insertId": « xxxxxxx »,
  "resource": {
    "type": "project",
    "labels": {
      "project_id": « xxxxxx »
    }
  },
  "timestamp": "2023-03-12Txxxxxx",
  "severity": "NOTICE",
  "logName": "projects/xxxxxxx/logs/cloudaudit.googleapis.com%2Factivity",
  "receiveTimestamp": "2023-03-12Txxxx"
}
senarvi commented 1 year ago

@cyr06 that log message, "AssignResourceToBillingAccount", comes from linking the billing account to the project. I checked by removing the billing and then adding it again to the project. I get the same kind of log message. Still no map in Lightroom. It's working normally for you now?

cyr06 commented 1 year ago

Yes it's still working for me. What can be different:

Sorry if I can't help further

senarvi commented 1 year ago

@cyr06 don't worry, I'm just dumbfounded myself too! I tried also using only the 3 necessary patches. No luck. I'm using Windows, but the OP is using Mac, so I guess that's not the problem either. I give up, but hopefully someone else can figure this out.

astuder commented 1 year ago

@cyr06 can you share your patched .lua files with me by email? I'd like to check whether a) they work with my Windows install, and if they do, b) investigate what's different compared to mine. adrian at adrianstuder dot com

astuder commented 1 year ago

For those playing along at home, I just figured out how to enable the debug log:

  1. In Lightroom, go to Help > System Info and find Settings Folder
  2. In that folder, create a new file named config.lua with the following text content:
    loggers["Location"] = {
    logLevel = 'trace',
    action = 'logfile',
    }
  3. Restart Lightroom and you will see log files in your user's Documents folder (Windows, might be somewhere else on Mac)

There are a few ERROR entries, though not sure how to interpret them. Log level trace can be replaced with debug for different output.

Tibashoult commented 1 year ago

Hi Astuder, I try your instruction. No new line in the file config.lua.

My setting location is "C:\Users\Thibault\AppData\Roaming\Adobe\Lightroom"

I restart few time, change picture, goes du development / map / .... nothing recorded in the file.

Attached my file (just add '.txt' to load attchment in this website. config.lua.txt

astuder commented 1 year ago

Copied your file, and Works For Me (TM)

The log files are created in C:\Users\[login]\Documents

Peter-MI commented 1 year ago

Copied your file, and Works For Me (TM)

The log files are created in C:\Users\[login]\Documents

Works for me as well. A click on location information flag goes to Map module and creates 4 log files Location....log

cyr06 commented 1 year ago

adrianstuder

done