openaustralia morph issues

openaustralia / morph

Take the hassle out of web scraping

https://morph.io

GNU Affero General Public License v3.0

461 stars 74 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Scraper failing with status code 255

#1086 henare opened 8 years ago
17
can't delete scrapers

#1085 coyotemarin closed 8 years ago
2
Cannot delete scraper due to foreign key constraint failing

#1084 henare closed 7 years ago
6
[Morph/production] ActionView::Template::Error: database disk image is malformed

#1083 henare closed 8 years ago
0
[Morph/production] ActionView::Template::Error: database disk image is malformed

#1082 henare closed 8 years ago
0
Limit log lines in process

#1081 mlandauer closed 8 years ago
2
Simplify connection_logs table to reduce space. Fixes #1079

#1080 mlandauer closed 8 years ago
1
Reduce disk space overhead of storing connection_logs information

#1079 mlandauer closed 8 years ago
0
Scrapers sometimes report they've scraped more pages than they actually do

#1078 wfdd opened 8 years ago
1
Rearchitect collection of console output

#1077 mlandauer opened 8 years ago
0
Starvation for short running scrapers

#1076 mlandauer opened 8 years ago
6
Container clean up script for removing containers associated with deleted runs

#1075 mlandauer opened 8 years ago
0
Ensure that we're never running more scrapers than we should

#1074 mlandauer closed 8 years ago
2
Make Docker network subnet configurable

#1073 henare opened 8 years ago
1
Dynamic concurrent scraper setting

#1072 mlandauer closed 8 years ago
1
Exception thrown when trying to create connection logs

#1071 henare opened 8 years ago
1
Add watched scrapers link to main menu

#1070 wfdd opened 8 years ago
1
Significantly reduce peak memory usage of sidekiq process

#1069 mlandauer closed 8 years ago
2
Monitor container usage with cadvisor and Prometheus

#1068 mlandauer opened 8 years ago
1
Remove separation of background and main app in new relic config

#1067 mlandauer closed 8 years ago
0
Even with big amounts of memory sidekiq process is likely getting killed for OOM

#1066 mlandauer closed 8 years ago
2
Record which runs are getting killed because they're running too long in the database

#1065 mlandauer opened 8 years ago
1
Database file not writable?

#1064 wfdd closed 6 years ago
64
Handling transferred scrapers

#1063 tmtmtmtm opened 8 years ago
0
Reduce size of production server when possible

#1062 mlandauer closed 8 years ago
3
Add extra guards against running too many containers at once

#1061 mlandauer closed 8 years ago
0
Remove sidekiq unique jobs

#1060 mlandauer closed 8 years ago
0
Upgrade server to KVM

#1059 mlandauer closed 8 years ago
1
Move over to using postgres as the database for the web application

#1058 mlandauer opened 8 years ago
3
Clean up old images by using a disk space limit

#1057 mlandauer closed 8 years ago
2
Set noeviction policy on redis

#1056 mlandauer closed 7 years ago
5
Runs not attached to scrapers not getting cleaned up properly

#1055 mlandauer closed 8 years ago
0
Only redirect traffic from morph docker containers to mitmproxy

#1054 mlandauer closed 8 years ago
1
Python connectivity issues

#1053 wfdd closed 8 years ago
3
In admin console "docker images" only show images that are the result of a compile

#1052 mlandauer opened 8 years ago
0
[Morph/production] ActionView::Template::Error: database disk image is malformed

#1051 henare closed 7 years ago
1
Scrapers can appear to run for more than 24 hours

#1050 henare closed 7 years ago
10
Mitmproxy fixes

#1049 mlandauer closed 8 years ago
1
[Morph/production] NameError: uninitialized constant RestClient::MaxRedirectsReached

#1048 henare closed 8 years ago
0
Small test fixes after updating buildstep

#1047 mlandauer closed 8 years ago
4
Fix referential integrity part 2

#1046 mlandauer closed 8 years ago
1
Remove need for intermediate images

#1045 mlandauer closed 8 years ago
2
Fix referential integrity part 1

#1044 mlandauer closed 8 years ago
2
Include scraper name in data passed in the webhook post request

#1043 mlandauer opened 8 years ago
0
Intermediate container cleanup should be done at the end

#1042 mlandauer closed 8 years ago
1
[Morph/production] Excon::Error::Socket: Mysql2::Error: Data too long for column 'text' at row 1: INSERT INTO `log_lines` (`run_id`, `timestamp`, `stream`, `text`, `created_at`, `updated_at`) VALUES (481387, '2016-07-26 23:09:40.757079', 'stderr', '{\'content\': u\'<div class=\"grid\"><div class=\"row m-b-xl\"><div class=\"col-1-1\"><div class=\"row\"><div class=\"l-col-10-12 xl-col-20-24\"><h1>Microsoft Privacy Statement</h1><div class=\"Lastupdated\"><span id=\"psp_last_updated\">Last Updated:</span><s...

#1041 henare closed 8 years ago
0
[Morph/production] ArgumentError: wrong number of arguments (given 2, expected 3)

#1040 henare closed 8 years ago
0
Limit log lines to 10,000 lines

#1039 mlandauer closed 8 years ago
1
Fix referential integrity in production database

#1038 mlandauer closed 8 years ago
1
Update server to mysql 5.6

#1037 mlandauer closed 8 years ago
0

Previous Next