search

MapServer / MapServer-import

3 stars 2 forks source link

Rasters are resampled to bad colours when zooming out #1566

Open tbonfort opened 12 years ago

tbonfort commented 12 years ago

Reporter: kenlord@gmail.com Date: 2005/12/12 - 05:53

Here's a copy of the message posted to the user's list, with more information 
below it:

I've just upgraded a site to MapServer 4.8beta2 from 4.4.1, using the windows binary available at maptools.org/dl

I'm seeing an raster resampling problem.

When I zoom out, and a larger part of raster data sources are resampled down to fit the map, it starts to resemble that old 256 colour limit in the mapserver 3.6 days with really wrong colours overtaking the rasters ... and only affecting the raster layers, the vector layers always draw fine.

I am using a 24bit output format.

The problem affects rasters layer by layer, having multiple raster layers turned on doesn't change how the individual rasters are affected.

I didnt see this problem at all using MapServer 4.4.1.

I've looked at a few different raster layers, some are satellite images, some are coloured geophysics images and all have the problem in a similar fashion whether they are 24bit, 8bit (256 colour), or 8bit (120 colour).

The only difference is that the 120 colour raster, which had been down sampled to display properly in mapserver 3.6, is not as badly affected. In this case some of the colours end up resampled to match my no-data offsite colour and just become transparent in the map, a coincidence of where the offsite colour bin falls within this raster's colour table I assume.

Is there a processing directive that will force resolution downsampling without colour-matching the remaining pixels?

Here are examples of a good and bad map ... I've set the background colour of the map to red to highlight the problem. The geophysics layer really gets hacked up in the bad one, loosing a few colours.

Good map: http://cariboo-gold.no-ip.info/IWA11342917155380.jpg

Bad map: http://cariboo-gold.no-ip.info/IWA11342916912848.jpg


More info:

I have found that the problem does not affect all rasters. They have come from 
different sources however they have been handled in a similar manner with Corel 
PhotoPaint10 for use with mapserver.  The problem is seen when using shp2img 
4.8beta1 as well (from FWTools 1.0.0a8).

A simplified version of my mapfile can be downloaded here, it contains notes on 
which raster layers work, and which dont, and two extents where the map is good 
and bad:

http://cariboo-gold.no-ip.info/MS/mapserver/testIWA.map

The data for the layers in this mapfile can be downloaded by anonymous ftp here:

ftp://cariboo-gold.no-ip.info

Some of the data sources are quite large, upto 200mb ... you may want to 
download welbarmag1.tif  and .tfw  first as this one is affected by the problem 
and is only 13mb.

Thanks,
Ken Lord
Vancouver BC
tbonfort commented 12 years ago

Author: fwarmerdam Date: 2005/12/12 - 08:19

Ken,

First, I couldn't fetch the .map you linked to.  For some
reason clicking on it returns me to this bugzilla bug report.

I tried mapserver CVS on linux, and FWTools 1.0.0a7 on win32
with a mapfile I constructed myself and it worked fine.   The
mapfile was:

MAP
  EXTENT 583665.485 5858724.540 613565.485 5895024.540
  SIZE 500 500 
  UNITS meters
  NAME mapname
  DEBUG ON

  IMAGETYPE jpeg

OUTPUTFORMAT
 NAME jpeg
 DRIVER "GD/JPEG"
 MIMETYPE "image/jpeg"
 IMAGEMODE RGB
 EXTENSION "jpg"
 FORMATOPTION "QUALITY=90"
END

LAYER
 NAME LANDSAT7FC
 TYPE RASTER
 STATUS DEFAULT
 DATA "welbarmag1.tif"
 OFFSITE 0 0 0
 PROJECTION
  "init=epsg:26910"
 END
END

END

The command I used with this was "shp2img -m test.map -o out.jpg"

I am attaching the result which seemed OK.
tbonfort commented 12 years ago

Author: kenlord@gmail.com Date: 2005/12/12 - 08:47

That is strange about the map file link,  The contents of my mapfile are shown 
below.

Your result does appear to be fine. I used the same command when i tried it 
with shp2img with FWTools 1.0.0a8.

If you visit cariboo-gold.no-ip.info and just zoom out a couple times, you will 
see it happen there in the in the landsat image that loads from the start ... 
brownish holes appear in the landsat, like the red holes in the example in the 
previous posts.  

The geophysics layer that you have tested is turned on by the first control 
(Airbourne - Mag) in the 'Geophysics Layers' folder in the layer controls panel 
(click the 'Layer Controls' on the top bar).

Here's my simplified map file:

####################
# The STATUS of the layers have been set to OFF, except for GPWELBARMAG, which 
uses welbarmag1.tif and exhibits the problem.
###################
MAP

NAME IWA
STATUS ON
SIZE 550 550

#This extent is zoomed out enough to show the problem in the GPWELBARMAG layer.
EXTENT 580000 5865000 610000 5895000

#At this extent, the problem does not occur with the GPWELBARMAG layer.
#The LANDSAT7FC layer is affected at similar changes of scale.
#EXTENT 587000 5872000 603000 5888000

UNITS meters
SHAPEPATH "data/IWA/"
IMAGECOLOR 255 255 255 #0 0 0

IMAGETYPE jpeg

OUTPUTFORMAT
  NAME jpeg
  DRIVER "GD/JPEG"  
  MIMETYPE "image/jpeg"
  IMAGEMODE RGB
  EXTENSION "jpg"
  FORMATOPTION "QUALITY=90"
END

FONTSET "fonts/fonts.txt"
SYMBOLSET "symbols/symbols.txt"
PROJECTION
  "init=EPSG:26910"
END

TEMPLATEPATTERN "IWA"

WEB
  #ERROR "../MS/mapserver/error.html"
  EMPTY "../MS/mapserver/results/empty_results.html"
  TEMPLATE void #"template.html"
  HEADER "results/results_header.html"
  MINSCALE 100
  MAXSCALE 50000000
  IMAGEPATH "C:/Inetpub/wwwroot/tmp/"
  IMAGEURL "/tmp/"
  LOG "C:/Inetpub/wwwroot/tmp/MS_LOG.txt"
  METADATA
    wms_title "Cariboo Gold Project"   
    wms_onlineresource "http://24.87.220.99/scripts/ms48b2/mapserv.exe?
map=c:/inetpub/wwwroot/ms/mapserver/testIWA.map&"
    wms_srs "EPSG:26909 EPSG:26910 EPSG:26911 EPSG:4326 EPSG:4269 EPSG:3005"
  END
END

SCALEBAR
  IMAGECOLOR 255 255 255
  LABEL
    COLOR 0 0 0 
    SIZE medium
    OUTLINECOLOR 255 255 255
  END
  SIZE 300 5
  COLOR 255 255 255
  BACKGROUNDCOLOR 0 0 0
  OUTLINECOLOR 0 0 0
  UNITS kilometers
  INTERVALS 5
  STATUS EMBED
  TRANSPARENT ON
END

############
# This layer is NOT affected by the resampling problem
############

LAYER
  NAME PROVINCE
  STATUS OFF
  TYPE RASTER
  DATA "BC_DEM30B1.tif"   #"Province1"
  REQUIRES "![Global_LandSat7_WMS]"  
  OFFSITE 60 80 100
  PROJECTION
    "init=EPSG:26910"
  END
  METADATA
     WMS_TITLE "580m Resolution Elevation DEM"
     WMS_ABSTRACT "580m resolution DEM, BCGS datasource"
     WMS_SRS "EPSG:26910"
  END
END 

############
# This layer is NOT affected by the resampling problem
############
LAYER
  NAME Global_LandSat7_WMS
  TYPE RASTER
  STATUS OFF
  CONNECTIONTYPE WMS
  CONNECTION "http://wms.jpl.nasa.gov/wms.cgi?"
  MAXSCALE 10000000
  #MINSCALE 20000
  METADATA
    "wms_name" "global_mosaic" 
    "wms_srs" "EPSG:4326"
    "wms_styles" "visual"
    "wms_server_version" "1.1.1"
    "wms_layers" "global_mosaic" 
    "wms_request" "GetMap"
    "wms_format" "image/jpeg"
    "wms_connectiontimeout" "30"
    "wms_title" "global_mosaic_from_NASA_WMS"
  END
END

############
# This layer IS affected by the resampling problem
############
LAYER
  NAME LANDSAT7FC
  TYPE RASTER
  STATUS ON
  DATA "LANDSAT7_15m_8bit_IWA.tif" 
  REQUIRES "![Global_LandSat7_WMS]"
  OFFSITE 0 0 0
  PROJECTION
   "init=EPSG:26910"
  END
  METADATA
     WMS_TITLE "LandSat7 15m Bands321"
     WMS_ABSTRACT "LandSat7"
     WMS_SRS "EPSG:26910"
  END
END 

############
# This layer is NOT affected by the resampling problem
############
LAYER
  NAME ORTHO4
  GROUP ORTHO
  TYPE RASTER
  STATUS OFF
  DATA "iw4_8bit121.tif" 
  OFFSITE 255 255 255
  PROJECTION
   "init=EPSG:26910"
  END
  #MAXSCALE 3000
  METADATA
     WMS_TITLE "0.25m Orthophoto Mosaic. Flown by Eagle Mapping 2000, 4 of 6."
     WMS_ABSTRACT "Ortho"
     WMS_SRS "EPSG:26910"
  END
END 

############
# This layer IS affected by the resampling problem
############
LAYER
  NAME GPWELBARMAG
  TYPE RASTER
  STATUS ON
  DATA "WelbarMAG1.tif" 
  OFFSITE 0 0 0
  PROJECTION
   "init=EPSG:26910"
  END
  METADATA
     WMS_TITLE "Regional -  Airborne MAG"
     WMS_ABSTRACT "No legend available, ~1995 Welbar Project, GoldCity 
Industries"
     WMS_SRS "EPSG:26910"
  END
END 

############
# This layer is NOT affected by the resampling problem
############
LAYER
  NAME GPWELBARVD1
  TYPE RASTER
  STATUS OFF
  DATA "WelbarVD1.tif" 
  OFFSITE 0 0 0
  PROJECTION
   "init=EPSG:26910"
  END
  METADATA
     WMS_TITLE "Regional -  Airborne VLF D1"
     WMS_ABSTRACT "No legend available, ~1995 Welbar Project, GoldCity 
Industries"
     WMS_SRS "EPSG:26910"
  END
END 

############
# This layer is NOT affected by the resampling problem
############
LAYER
  NAME GPWELBARVD2
  TYPE RASTER
  STATUS OFF
  DATA "WelbarVLF2.tif" 
  OFFSITE 0 0 0
  PROJECTION
   "init=EPSG:26910"
  END
  METADATA
     WMS_TITLE "Regional -  Airborne VLF D2"
     WMS_ABSTRACT "No legend available, ~1995 Welbar Project, GoldCity 
Industries"
     WMS_SRS "EPSG:26910"
  END
END 

END # Map File
tbonfort commented 12 years ago

Author: fwarmerdam Date: 2005/12/12 - 17:04

Ken,

Can you boil things down to a minimum example demonstrating the problem?
tbonfort commented 12 years ago

Author: kenlord@gmail.com Date: 2005/12/12 - 18:56

OK, I'll try to make a simpler example at home tonight, for now try the 
following link ... copy it into your browser if Bugzilla won't take you to it. 
This example uses the same settings found in the map file posted above, 
although in this case is the full map file for my website.

... It's set at a scale that shows the problem, zoom in once and you'll see a 
good map.

http://cariboo-gold.no-ip.info/scripts/ms48b2/mapserv.exe?mode=browse&zoomdir=-
1&zoomsize=2&imgext=586844.825300+5873883.505170+602353.417828+5889392.097698&m
ap=c%3A%5CInetpub%5Cwwwroot%5CMS%5Cmapserver%
5CIWA.map&savequery=true&program=..%2F..%2Fscripts%2Fms48b2%
2Fmapserv.exe&map_web_template=IWA_html_template.html&map_web_imagepath=c%3A%
5CInetpub%5Cwwwroot%5Ctmp%5C&map_web_imageurl=%2Ftmp%
2F&elerank=au_rank&mapext=586844.825300+5873883.505170+602353.417828+5889392.09
7698&template=%5Btemplate%5D&INPUT_TYPE=&INPUT_COORD=&lastButtonSelected=%
5BlastButtonSelected%5D&layer=GPWELBARMAG&layer=LANDSAT7FC&img.x=243&img.y=267

Thanks,
Ken
tbonfort commented 12 years ago

Author: kenlord@gmail.com Date: 2005/12/12 - 19:00

Bugzilla is not so friendly to links ...

I've emailed the link directly to you.

Ken
tbonfort commented 12 years ago

Author: fwarmerdam Date: 2005/12/12 - 19:18


Ken provides the following really long url to demonstrate
the problem:

http://cariboo-gold.no-ip.info/scripts/ms48b2/mapserv.exe?mode=browse&zoomdir=-1&zoomsize=2&imgext=586844.825300+5873883.505170+602353.417828+5889392.097698&map=c%3A%5CInetpub%5Cwwwroot%5CMS%5Cmapserver%5CIWA.map&savequery=true&program=..%2F..%2Fscripts%2Fms48b2%2Fmapserv.exe&map_web_template=IWA_html_template.html&map_web_imagepath=c%3A%5CInetpub%5Cwwwroot%5Ctmp%5C&map_web_imageurl=%2Ftmp%2F&elerank=au_rank&mapext=586844.825300+5873883.505170+602353.417828+5889392.097698&template=%5Btemplate%5D&INPUT_TYPE=&INPUT_COORD=&lastButtonSelected=%5BlastButtonSelected%5D&layer=GPWELBARMAG&layer=LANDSAT7FC&img.x=243&img.y=267

From this the problem would appear to be some error in offsite processing with
quite a few data values all getting treated as offsite (or at least transparent
after some fashion).
tbonfort commented 12 years ago

Author: fwarmerdam Date: 2005/12/12 - 19:58

Ken,

I downloaded FWTools 1.0.0a8 and tested with it against roughly your map
with just the welbarmag1.tif data available and your url.  I have zipped
up the files I used (running test.bat) into a zip file.  The zip file has
everything *except* the welbarmag1.tif and welbarmag1.tfw files which are
pretty big.  note that this test did not include the LANDSAT data so debug
messages report it was not found.  Perhaps it is required to reproduce the 
problem, but I'm leery to download the 200MB+ just for a quick test. 

Could you grab the zip file, test at your end, and then modify that 
configuration till it demonstrates the error?  If I need to download the
landsat data then I'll do so, though if you find the problem can be reproduced
with a reduced resolution version then that would be better.
tbonfort commented 12 years ago

Author: kenlord@gmail.com Date: 2005/12/12 - 22:21

Frank, I've used your test setup from the zip file, with both the landsat7fc 
layer and the welbarmag1 layer, using FWTools 1.0.0a8 (downloaded and 
installed on this machine today)

From my work computer it is working fine, there's no problem with the images.

Tonight I'll try it again on my computer at home.

I'm not sure why they would output differently, both working through the 
FWTools shell, windows XP, same map file etc.

My home computer does have mapserver 4.4.1 as well as 4.8b2, but the dll's are 
isolated in their own folders.
tbonfort commented 12 years ago

Author: kenlord@gmail.com Date: 2005/12/13 - 18:48

I tried the test setup on my home computer, where the problem is.  Using the 
files in the test setup there is no problem, the images generate properly.

I can't see a difference between the test files and my main map file, or 
anywhere else for that matter so I don't know why it is acting differently.  
Running the test files with my main mapfile appears to be ok as well,  But, 
the website still has the problem.

I've further isolated the older version of mapserver on my system, renaming 
the folder.

I copied the layers blocks out of the test map file into my main mapfile ... 
even though they appear identical ... and oddly it did make some difference, 
you have to zoom out a bit more for the problem to happen.

So I'm pretty confused, and don't know where we can go from here with the bug.
tbonfort commented 12 years ago

Author: fwarmerdam Date: 2005/12/14 - 05:25

Ken,

I'm not sure what to do either.  Till you can reduce this to a case
I can reproduce with mapserv.exe, there isn't much I can do.
tbonfort commented 12 years ago

Author: kenlord@gmail.com Date: 2005/12/14 - 06:18

Ok, Thanks very much for your efforts on this, If I can wrangle any more clues 
out of my system, I'll post them.