layer file symlinks are not being updated

[springmeyer]

On OSX 10.6.8, tilemill master @ 2b2794d8.

1. Create a new project called "intro"
2. Delete the shapefile layer, create an sqlite layer
3. Save the layer and the project - works.
4. Go back to edit layer, change the * Datasource input line to point to a different sqlite file
5. Save the layer and the project

At this point rendering still uses the old layer. The reason for this is visible by viewing the Mapnik XML at

http://localhost:8889/api/Project/intro.xml

The XML shows:

Which indicates a symlink is pointed to for accessing the layer, and that symlink is the same name (did not change), and that symlink points to the same original filename.

$ pwd
/Users/dane/Documents/MapBox/project/intro/layers
d:layers dane$ ls -l
total 24
lrwxr-xr-x  1 dane  staff  49 Aug 17 08:42 c.sqlite -> /Users/dane/Desktop/demo/project/countries.sqlite
lrwxr-xr-x  1 dane  staff  65 Aug 17 08:33 countries -> /Users/dane/Documents/MapBox/cache/1aa2d236-10m-admin-0-countries
lrwxr-xr-x  1 dane  staff  46 Aug 17 08:33 w.sqlite -> /Users/dane/Documents/MapBox/data/world.sqlite

Curiously the mml file is pointing to the right sqlite file "countries_dane.sqlite" vs "countries.sqlite":

$ cat ../intro.mml | grep "file"
        "file": "/Users/dane/Desktop/demo/project/countries2.sqlite",

And here are the details on those files:

$ ls -l /Users/dane/Desktop/demo/project/countries*
-rw-r--r--@ 1 dane  wheel  8810496 Aug 16 10:11 /Users/dane/Desktop/demo/project/countries.sqlite
-rw-r--r--@ 1 dane  wheel  8785920 Aug 17 08:59 /Users/dane/Desktop/demo/project/countries_dane.sqlite

[springmeyer]

In fact, I can create any filename I like the the layer UI, the mml is updated, and the project rendering does not change:

$ cat ../intro.mml | grep "file"
        "file": "/Users/dane/Desktop/demo/project/thisdoesnotexist.sqlite",

[springmeyer]

okay, I see now that you have to update the layer #id to get the symlinks to update. -1 on this behavior.

[yhahn]

I will have to look at this more closely in millstone... I know that there are some limitations to what we can do here.

For example, it's not actually possible to efficiently know when a symlink is out of date for URL zip resources because you would have to download and unzip the resource every time, eliminating the usefulness of localization/caching.

For absolute files on the filesystem, however, it may be reasonable to check symlinks every time to make sure they point at the right place.

[springmeyer]

Yes, local filesystem paths are the issue here which I think need fixing. Having to change the #id requires changing both your layer and your style and litters your project folder with dead symlinks.

[yhahn]

Should be fixed with https://github.com/mapbox/millstone/commit/0b21f2f76a244eb4de08e876314d997604c867b0

I tested with two local sqlite datasources as well as running the tests but would love confirmation from someone that the fix above works.

[springmeyer]

tested, this is fixed! thank you!

[willwhite]

Awesome, it works!