wikimaps-dev / mapwarper

Wikimedia Commons map georectifier and warper. See wikimaps_new branch.
http://warper.wmflabs.org
11 stars 1 forks source link

Error when adding maps from commons with long title / desc #77

Closed timwaters closed 8 years ago

timwaters commented 8 years ago

E, [2016-02-09T15:22:03.969891 #27356] ERROR -- : PG::StringDataRightTruncation: ERROR: value too long for type character varying(255) : INSERT INTO "maps" ("created_at", "description", "image_url", "import_id", "map_type", "mask_status", "owner_id", "page_id", "rough_state", "source_uri", "status", "title", "unique_id", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14) RETURNING "id" When importing https://commons.wikimedia.org/wiki/Category:Old_maps_of_Helsinki

74 maps, some of which have long titles (larger than 255 characters)

E, [2016-02-09T15:22:03.970701 #27356] ERROR -- : Error with import #<ActiveRecord::StatementInvalid: PG::StringDataRightTruncation: ERROR: value too long for type character varying(255) : INSERT INTO "maps" ("created_at", "description", "image_url", "import_id", "map_type", "mask_status", "owner_id", "page_id", "rough_state", "source_uri", "status", "title", "unique_id", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14) RETURNING "id"> E, [2016-02-09T15:22:03.970818 #27356]

timwaters commented 8 years ago

https://www.mediawiki.org/wiki/Page_title_size_limitations

So the title should fit within a postgres varchar /ruby string but it's when we calculate the image url and store this that we are seeing problems.

timwaters commented 8 years ago

Would need to change image_url only I think from string to text

timwaters commented 8 years ago

I've copied one of the helsinki maps to beta commons: http://commons.wikimedia.beta.wmflabs.org/wiki/File:Plan_Charta_%C3%B6fver_Helsingfors_Stad_Utvisande_s%C3%A5_v%C3%A4l_dess_%C3%A4nnu_varande_gamla_och_och_Qvarter_och_Gator_som_%C3%A4fven_den_nya_af_Hans_Kejserliga_Majest%C3%A4t_i_N%C3%A5der_approberade_Reglering_2.jpeg

and this causes the error

PG::StringDataRightTruncation: ERROR: value too long for type character varying(255) : INSERT INTO "maps" ("created_at", "description", "height", "image_url", "map_type", "mask_status", "owner_id", "page_id", "rough_state", "source_uri", "status", "title", "unique_id", "updated_at", "upload_content_type", "upload_file_name", "upload_file_size", "width") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18) RETURNING "id" Extracted source (around line #94):

respond_to do |format|
  if @map.save
    flash[:notice] = 'Map was successfully created.'
    format.html { redirect_to map_path(@map.id) }
  else
timwaters commented 8 years ago

This should be fixed in commit d2b0822