mapcache windows build - Githubissues

tbonfort commented 12 years ago

Reporter: assefa Date: 2011/12/02 - 18:21 mapcache windows build

tbonfort commented 12 years ago

Author: assefa Date: 2011/12/02 - 18:25 Thomas,

Is it possible to apply the patch and tell me if things still work. I can then commit. The patch is only the first pass to bring the code to build on windows.

tbonfort commented 12 years ago

Date: 2011/12/02 - 18:33 Assefa, I have applied the patch and all seems correct on my osx build.

I skimmed through the patch, can you confirm that it only moves the variable declarations to the beginning of their code block?

tbonfort commented 12 years ago

Author: assefa Date: 2011/12/02 - 19:01 majority of the change is that. but I also added a bit of things using ifdefs (#ifndef _WIN32) for things that did not build properly on windows (ex buffer.c, ezxml.c, http.c, imageio_png.c) look for any_WIN32 to see them. I think it should be ok but can you give it a quick look at least for buffer.c

tbonfort commented 12 years ago

Date: 2011/12/02 - 19:15 Assefa, seems good to me, thanks for fixing those, I'll try to be more careful in the future now you windows guys are poking around in the code :)

some of the _WIN32 ifdefs also be applied to the unix builds, but we can do that in a next step.

tbonfort commented 12 years ago

Author: assefa Date: 2011/12/02 - 19:23 committed in r 12835. I will start committing the make files. It won't be complete though.

PS: I am an ubuntu guy since FOSS4G :)

tbonfort commented 12 years ago

Author: rouault Date: 2011/12/02 - 20:07 I humbly suggest to turn (trunk/mapserver/mapcache/src/buffer.c) :

#ifndef _WIN32 
    memcpy(buffer->buf + buffer->size, data, len); 
#else 
    memcpy(((unsigned char*)buffer->buf) + buffer->size, data, len); 
#endif

into :

memcpy(((unsigned char*)buffer->buf) + buffer->size, data, len);

(or just (char*))

which should work on all platforms. void* pointer arithmetic compiled with -pedantic raises a warning.

tbonfort commented 12 years ago

Author: assefa Date: 2011/12/02 - 23:09 I have done the change in buffer.c according to comment. 65a465ffdd79a53a722df80ab008e7ead57e3742 (r12838)

windows make files committed:

build the fcgi
seed executable does not yet build: missing time related functions
did not try yet the Apache module

Note for Jeff:

I used : http://apache.sunsite.ualberta.ca//apr/binaries/win32/
http://apr.apache.org/download.cgi to build the modules locally. I started with that but the 2nd module did not build out of the box. I did not make any efforts on building it.
You can modify/fix ad you see fit

tbonfort commented 12 years ago

Date: 2011/12/03 - 09:01 Assefa,

what are the time functions that are failing? All time handling is supposed to be done through apr which should take care of platform idiosyncrasies.

tbonfort commented 12 years ago

Author: assefa Date: 2011/12/03 - 14:38 It is not that they are failing, It is just that they are not available on windows:

strptime
gettimeofday

We need to define them locally as it is done for MapServer-cgi.

The other link problem I had was with:

mapcache_seed.obj : error LNK2019: unresolved external symbol __imp__apr_time_an
si_put@12 referenced in function _main

I am not sure why it was not available in the pre-build binaries I used. Probably need to build a local copy of the apr libraries.

tbonfort commented 12 years ago

Author: jmckenna Date: 2011/12/06 - 22:37 I have compiled from source (apr-1.4.5, apr-iconv-1.2.1, apr-util-1.3.12) and I now only get one error when I execute 'nmake /f makefile.vc' for mapcache, but I am not sure which library this function/symbol comes from:

fastcgi_mapcache.obj : error LNK2019: unresolved external symbol __imp__apr_date_parse_http@4 referenced in function _fcgi_write_response
mapcache.exe : fatal error LNK1120: 1 unresolved externals

tbonfort commented 12 years ago

Author: assefa Date: 2011/12/06 - 23:22 It should be in apr-util (source code I see apr-util-1.3.12\misc\apr_date.c(145))

tbonfort commented 12 years ago

Author: jmckenna Date: 2011/12/07 - 00:00 ok thanks, I think I am missing libaprutil-1.lib and it might be causing this.

tbonfort commented 12 years ago

Author: jmckenna Date: 2011/12/07 - 00:17 I think I made it past that. 2 new missing symbols:

mapcache_seed.obj : error LNK2019: unresolved external symbol _strptime referenced in fun
tion _main
mapcache_seed.obj : error LNK2019: unresolved external symbol _gettimeofday referenced in
function _main
mapcache_seed.exe : fatal error LNK1120: 2 unresolved externals

tbonfort commented 12 years ago

Author: aboudreault Date: 2011/12/21 - 18:04 I talked with Assefa, and I doubt he'll get time to finish this task. I'm going to try take it. Question: how did you get it to compile ... I'm getting a lot of illegal declaration errors. (due to a declaration not at the beginning of a scope. Didn't find the c99 flag for VC 2010 and VC 2008.

tbonfort commented 12 years ago

Author: aboudreault Date: 2011/12/21 - 20:51 nvm about the declaration errors... I think my mapcache directory was not up-to-date.

tbonfort commented 12 years ago

Author: jmckenna Date: 2011/12/21 - 23:25 I'm not sure if you are talking to me directly - but I left this since there were 2 missing functions in the mapcache code, not part of MSVC/SDK (see comment:13) I believe there are already workarounds for these in the MS base code, and I talked to Assefa about this also. I have been meaning to dive back in and port those 2 functions over, but, haven't yet :)

Short story: I never did get it to compile.

tbonfort commented 12 years ago

Author: assefa Date: 2011/12/22 - 05:09 I have ported locally the code from MS to mapcahe for the 2 functions but did not commit yet. I have not done the module building yet and not sure when I will get to this.

tbonfort commented 12 years ago

Author: aboudreault Date: 2011/12/22 - 14:49 Assefa, Please attach a patch of you changes not committed, I would like to try building/porting the mapcache module.

tbonfort commented 12 years ago

Author: jmckenna Date: 2011/12/22 - 15:24 same comment here :)

tbonfort commented 12 years ago

Author: assefa Date: 2011/12/23 - 17:46 It is committed ( 5e5cde4b5dba9a1e0a3b29ac162b639f107081e4 (r12915) and 809508117c7d611f10b7576c44a7ccb45dcf6cc3 (r12916))

tbonfort commented 12 years ago

Author: aboudreault Date: 2012/01/10 - 18:11 I've just committed some changes to be able to build mapcache under Windows in 0e1b04f78103a96d151df9016b118ad75228c4a6 (r12980). Everything builds now. I have been able to make a mod_mapcache.dll for OSGeo4w and it loads properly. Unfortunately I'm getting this error when trying to access a tile (which do not exist yet):

failed to open file C:/OSGeo4W/tmp/test/g/01/000/000/000/000/000/001.png: Unknown error

Not sure what this error means. Nothing is created in my cache base path.

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/12 - 21:12 I am able to compile mod_mapcache.dll, mapcache.exe, and mapcache_seed.exe but have not tested yet.

tbonfort commented 12 years ago

Author: aboudreault Date: 2012/01/12 - 22:20 Jeff, if you test the mod_mapcache.dll... and it works... let me know. I'm still working on this... looks like something is different under windows and it's tricky to debug.

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/13 - 18:30

committed changes to makefile.vc (e475080a85afde25e57804d1ffe587c6a4777f6a (r12982)) for embedding the manifest files in the exes and dll
haven't tested other stuff yet

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/13 - 21:54 More feedback:

after configuring httpd.conf, I received an error:

    D:\ms4w>apache-restart.bat

      Syntax error on line 375 of D:/ms4w/Apache/conf/httpd.conf:
      cache disk: host system does not support file symbolic linking

workaround I used was to comment in mapcache.xml

          <cache name="disk" type="disk">
            <base>D:/ms4w/tmp/ms_tmp/cache</base>
            <!--<symlink_blank/>-->
          </cache>

I cannot load mapcache_module as a Windows service for some unknown reason:

The Apache MS4W Web Server service is starting. The Apache MS4W Web Server service could not be started.

A service specific error occurred: 1.

More help is available by typing NET HELPMSG 3547.

I am able to verify that the module can be loaded:


   httpd -t -D DUMP_MODULES

... negotiation_module (shared setenvif_module (shared) mapcache_module (shared)

if I manually start httpd.exe through the command window I can in fact load the module (????)
if I do that, I goto http://127.0.0.1/mapcache/demo/tms and the following is recorded in Apache's error.log


[Fri Jan 13 16:36:54 2012] [error] [client 127.0.0.1] received wms request with no service param
[Fri Jan 13 16:37:07 2012] [error] [client 127.0.0.1] failed to open file D:/ms4w/tmp/ms_tmp/cache/test/WGS84/01/000/000/000/000/000/000.png: Unknown error, referer: http://127.0.0.1/mapcache/demo/tms
[Fri Jan 13 16:37:07 2012] [error] [client 127.0.0.1] failed to open file D:/ms4w/tmp/ms_tmp/cache/test/WGS84/01/000/000/001/000/000/000.png: Unknown error, referer: http://127.0.0.1/mapcache/demo/tms
[Fri Jan 13 16:37:07 2012] [error] [client 127.0.0.1] failed to open file D:/ms4w/tmp/ms_tmp/cache/test/WGS84/01/000/000/002/000/000/000.png: Unknown error, referer: http://127.0.0.1/mapcache/demo/tms

for the record, I am using the default mapcache.xml file, pasted here:

<?xml version="1.0" encoding="UTF-8"?>


<!-- see the accompanying mapcache.xml.sample for a fully commented configuration file -->

<mapcache>
   <cache name="disk" type="disk">
      <base>D:/ms4w/tmp/ms_tmp/cache</base>
      <!--<symlink_blank/>-->
   </cache>

   <source name="vmap0" type="wms">
      <getmap>
         <params>
            <FORMAT>image/png</FORMAT>
            <LAYERS>basic</LAYERS>
         </params>
      </getmap>

http://vmap0.tiles.osgeo.org/wms/vmap0


   </source>

   <tileset name="test">
      <source>vmap0</source>
      <cache>disk</cache>
      <grid>WGS84</grid>
      <grid>g</grid>
      <format>PNG</format>
      <metatile>5 5</metatile>
      <metabuffer>10</metabuffer>
      <expires>3600</expires>
   </tileset>

   <default_format>JPEG</default_format>

   <service type="wms" enabled="true">
      <full_wms>assemble</full_wms>
      <resample_mode>bilinear</resample_mode>
      <format>JPEG</format>
      <maxsize>4096</maxsize>
   </service>
   <service type="wmts" enabled="true"/>
   <service type="tms" enabled="true"/>
   <service type="kml" enabled="true"/>
   <service type="gmaps" enabled="true"/>
   <service type="ve" enabled="true"/>
   <service type="demo" enabled="true"/>

   <errors>report</errors>
   <lock_dir>/tmp</lock_dir>

</mapcache>

tbonfort commented 12 years ago

Author: aboudreault Date: 2012/01/16 - 17:16 The commit ccbf323cdaaaa49e43c2efa8c2e5ba53e8659b3c (r12987) fixes the Unknown error issue.... but I'm still getting a crash in _mapcache_source_wms_render_map(). Unfortunately... it looks like debugging on Windows is just a mess. Cannot get a clear debug info where the module really crashes. In Apache, you should get a connection reseet by peer. Do you have the same thing on your machine Jeff?

tbonfort commented 12 years ago

Author: aboudreault Date: 2012/01/16 - 18:10 the crash is always in apr_snprintf.c, at line 1341 with an access violation. The backtrace I have is inconsistent and not very clear.

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/17 - 18:39 I fixed my service issue (now I can load mod-mapcache as a windows service). phew what a battle.

will keep testing.

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/17 - 18:52 Alan: I think I am getting your same httpd.exe crash. From the Windows Event logs:

  <System>
    <Provider Name="Application Error" /> 
    <EventID Qualifiers="0">1000</EventID> 
    <Level>2</Level> 
    <Task>100</Task> 
    <Keywords>0x80000000000000</Keywords> 
    <TimeCreated SystemTime="2012-01-17T17:48:26.000Z" /> 
    <EventRecordID>81022</EventRecordID> 
    <Channel>Application</Channel> 
    <Computer>kordic</Computer> 
    <Security /> 
  </System>
 <EventData>
   <Data>httpd.exe</Data> 
   <Data>2.2.21.0</Data> 
   <Data>4e6a3015</Data> 
   <Data>libapr-1.dll</Data> 
   <Data>1.4.5.0</Data> 
   <Data>4e6a2fa6</Data> 
   <Data>c0000005</Data> 
   <Data>0001101c</Data> 
   <Data>1044</Data> 
   <Data>01ccd5403687a124</Data> 
   </EventData>
 </Event>

tbonfort commented 12 years ago

Author: aboudreault Date: 2012/01/17 - 20:21 yep. same issue. Still unable to debug something. I don't know if it's my virtual machine that is broken somehow but I'm getting upset :| Let's see if Assefa can debug anything in it's VM.

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/17 - 20:44 my crash always on line#971 of apr_snprintf.c:

  line#971 s = va_arg(ap, char *);

specifically that "ap" value seems to be the problem:

  CXX0030: Error: expression cannot be evaluated

tbonfort commented 12 years ago

Author: aboudreault Date: 2012/01/17 - 20:54 same function than me...

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/17 - 21:11 Is the related call stack helpful?

    libapr-1.dll!apr_vformatter(int (apr_vformatter_buff_t *)* flush_func=0x6eec8710, apr_vformatter_buff_t * vbuff=0x00fcfd34, const char * fmt=0x1008ed7d, char * ap=0x20000000)  Line 971    
    libapr-1.dll!apr_pvsprintf(apr_pool_t * pool=0x030d2080, const char * fmt=0x1008ed68, char * ap=0x00000000)  Line 1142 + 0x16 bytes 
    mod_mapcache.dll!apache_context_request_log(mapcache_context * c=0x030d3d58, mapcache_log_level level=MAPCACHE_DEBUG, char * message=0x1008ed68, ...)  Line 148 + 0x15 bytes    
    mod_mapcache.dll!mapcache_tileset_tile_get(mapcache_context * ctx=0x030d3d58, mapcache_tile * tile=0x030e00b8)  Line 567 + 0x30 bytes   
    mod_mapcache.dll!mapcache_prefetch_tiles(mapcache_context * ctx=0x030d3d58, mapcache_tile * * tiles=0x030d4050, int ntiles=1)  Line 93 + 0x13 bytes 
    mod_mapcache.dll!mapcache_core_get_tile(mapcache_context * ctx=0x030d3d58, mapcache_request_get_tile * req_tile=0x030d4038)  Line 209 + 0x17 bytes  
    mod_mapcache.dll!mod_mapcache_request_handler(request_rec * r=0x030d20c0)  Line 293 + 0xd bytes 
    libhttpd.dll!ap_run_handler(request_rec * r=0x030d20c0)  Line 158 + 0x21 bytes  
    libhttpd.dll!ap_invoke_handler(request_rec * r=0x1008e11c)  Line 376 + 0x6 bytes    
    libhttpd.dll!ap_process_request(request_rec * r=0x030d20c0)  Line 286   
    libhttpd.dll!ap_process_http_connection(conn_rec * c=0x009e89c0)  Line 190 + 0x6 bytes  
    libhttpd.dll!ap_run_process_connection(conn_rec * c=0x009e89c0)  Line 43 + 0x21 bytes   
    libhttpd.dll!ap_process_connection(conn_rec * c=0x009e89c0, void * csd=0x009e8868)  Line 190 + 0x6 bytes    
    libhttpd.dll!worker_main(void * thread_num_val=0x009e89b8)  Line 784

tbonfort commented 12 years ago

Author: aboudreault Date: 2012/01/17 - 22:22 Thanks to you jmckenna! I've been able to fix the windows crash with your nice call stack! Fixed the last issue in 06ba2e459fdb1c4e5be892e0985f512b007ae386 (r12992). I haven't tested yet if the log reporting was working as expected yet but should do it early tomorrow.

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/17 - 23:02 Great. phew, that was hard work building an environment for that trace.

I verify that the fix removed the crash - I am watching my disk cache get generated through browser requests, on Windows!

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/19 - 18:50 sqlite cache seems very slow, pretty unusable I imagine. Works though.

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/19 - 19:35 tiff cache works (with Alan's e74f090f46df5f937fc427a1a30faba0d57a492d (r13003) changes).

tbonfort commented 12 years ago

Author: jmckenna Date: 2012/01/19 - 20:54 I don't have good news with my fcgi attempts with mapcache:

[Thu Jan 19 15:47:06 2012] [warn] [client 127.0.0.1] mod_fcgid: stderr: missing a service
[Thu Jan 19 15:47:06 2012] [error] [client 127.0.0.1] Premature end of script headers: mapcache.exe

My feeling is that mapcache-windows can be released now with the Apache module working, and come back to this later.

MapServer / mapcache

mapcache windows build #10