search

ngds / install-and-run

Use this repository's issue tracker to post comments, bug reports, and help questions on installing and running NGDS CKAN.
4 stars 3 forks source link

Internal Server Errors on new Amazon-hosted GINstack install #26

Closed ccaudill closed 9 years ago

ccaudill commented 9 years ago

I built a new instance on Amazon and test loaded a couple of CSV datasets. I keep getting messages about an “Internal Server Error” so I not sure if things are going at all right. The IP for the new server is: 52.5.202.58 Is there any chance you can see what is wrong and let me know what I should do next? The error crops up anytime I try to include or edit metadata.

per @dano-reisys:
Can you tell me what instance type they are using (it should be a minimum of 3 GB of ram for harvesting). Also, what centos distribution did they use? Behind that internal error there is a python fatal error that should give us more insight on this issue (/var/log/httpd/ckan.error.log)

ccaudill commented 9 years ago

This was on an Amazon EC2 instance. I'm not sure the size that was chosen; should we indicate that a t2.medium is needed? http://aws.amazon.com/ec2/instance-types/

ccaudill commented 9 years ago

Error logs, received by email:

[root@ip-172-31-34-93 httpd]# dir access_log ckan.custom.log-20150524 ckan.error.log-20150524 datapusher.error.log error_log-20150512 ckan.custom.log ckan.error.log datapusher.custom.log datapusher.error.log-20150513 error_log-20150517 ckan.custom.log-20150512 ckan.error.log-20150512 datapusher.custom.log-20150513 datapusher.error.log-20150517 error_log-20150524 ckan.custom.log-20150517 ckan.error.log-20150517 datapusher.custom.log-20150517 error_log [root@ip-172-31-34-93 httpd]# less ckan.error.log-20150512 [Tue May 12 14:05:30 2015] [error] 2015-05-12 14:05:30,787 INFO [ckan.lib.base] /user/admin render time 0.349 seconds [Tue May 12 14:05:31 2015] [error] 2015-05-12 14:05:31,020 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:05:43 2015] [error] 2015-05-12 14:05:43,110 INFO [ckan.lib.base] /ckan-admin render time 0.093 seconds [Tue May 12 14:05:43 2015] [error] 2015-05-12 14:05:43,344 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:05:49 2015] [error] 2015-05-12 14:05:49,146 INFO [ckan.lib.base] /pages render time 0.237 seconds [Tue May 12 14:05:49 2015] [error] 2015-05-12 14:05:49,385 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:05:53 2015] [error] 2015-05-12 14:05:53,742 INFO [ckan.lib.base] /user/admin render time 0.126 seconds [Tue May 12 14:05:53 2015] [error] 2015-05-12 14:05:53,973 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:06:03 2015] [error] 2015-05-12 14:06:03,085 INFO [ckan.lib.base] /user/admin render time 0.120 seconds [Tue May 12 14:06:03 2015] [error] 2015-05-12 14:06:03,334 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:06:09 2015] [error] 2015-05-12 14:06:09,937 INFO [ckan.lib.base] /user/edit/admin render time 0.205 seconds [Tue May 12 14:06:10 2015] [error] 2015-05-12 14:06:10,172 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:07:09 2015] [error] 2015-05-12 14:07:09,669 INFO [ckan.lib.base] /user/edit/admin render time 0.026 seconds [Tue May 12 14:07:10 2015] [error] 2015-05-12 14:07:10,025 INFO [ckan.lib.base] /user/admin render time 0.196 seconds [Tue May 12 14:07:10 2015] [error] 2015-05-12 14:07:10,274 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:07:57 2015] [error] 2015-05-12 14:07:57,972 INFO [ckan.lib.base] / render time 0.087 seconds [Tue May 12 14:07:58 2015] [error] 2015-05-12 14:07:58,963 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:08:01 2015] [error] 2015-05-12 14:08:01,299 INFO [ckan.lib.base] /user/login render time 0.209 seconds [Tue May 12 14:08:01 2015] [error] 2015-05-12 14:08:01,611 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:08:16 2015] [error] 2015-05-12 14:08:16,013 INFO [ckan.lib.base] /user/logged_in render time 0.004 seconds [Tue May 12 14:08:16 2015] [error] 2015-05-12 14:08:16,215 INFO [ckan.lib.base] /user/logged_in render time 0.051 seconds [Tue May 12 14:08:16 2015] [error] 2015-05-12 14:08:16,424 CRITI [ckan.logic] Action user_followee_list is being called directly all action calls should be accessed via logic.get_action [Tue May 12 14:08:16 2015] [error] 2015-05-12 14:08:16,430 CRITI [ckan.logic] Action dataset_followee_list is being called directly all action calls should be accessed via logic.get_action [Tue May 12 14:08:16 2015] [error] 2015-05-12 14:08:16,437 CRITI [ckan.logic] Action group_followee_list is being called directly all action calls should be accessed via logic.get_action [Tue May 12 14:08:16 2015] [error] 2015-05-12 14:08:16,443 CRITI [ckan.logic] Action dashboard_activity_list is being called directly all action calls should be accessed via logic.get_action [Tue May 12 14:08:16 2015] [error] 2015-05-12 14:08:16,557 INFO [ckan.lib.base] /dashboard render time 0.183 seconds [Tue May 12 14:08:16 2015] [error] 2015-05-12 14:08:16,872 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:14:36 2015] [error] 2015-05-12 14:14:36,789 INFO [ckan.lib.base] /user/admin render time 0.126 seconds [Tue May 12 14:14:37 2015] [error] 2015-05-12 14:14:37,031 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:14:40 2015] [error] 2015-05-12 14:14:40,733 INFO [ckan.lib.base] /user/_logout render time 0.005 seconds [Tue May 12 14:14:41 2015] [error] 2015-05-12 14:14:41,089 INFO [ckan.lib.base] /user/logged_out render time 0.005 seconds [Tue May 12 14:14:41 2015] [error] 2015-05-12 14:14:41,337 INFO [ckan.lib.base] /user/logged_out_redirect render time 0.070 seconds [Tue May 12 14:14:41 2015] [error] 2015-05-12 14:14:41,562 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Tue May 12 14:14:44 2015] [error] 2015-05-12 14:14:44,590 INFO [ckanext.facets.helpers] Metadata plugin custom facets loaded [Tue May 12 14:14:44 2015] [error] 2015-05-12 14:14:44,820 INFO [ckan.lib.base] /dataset render time 0.234 seconds [Tue May 12 14:14:45 2015] [error] 2015-05-12 14:14:45,073 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds (END)

[root@ip-172-31-34-93 httpd]# less ckan.error.log-20150517 [Fri May 15 14:07:54 2015] [error] 2015-05-15 14:07:54,690 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Fri May 15 14:07:58 2015] [error] [client 146.76.12.60] Error - <type 'exceptions.ValueError'>: invalid literal for int() with base 10: '' [Fri May 15 14:07:58 2015] [error] 2015-05-15 14:07:58,866 INFO [ckan.lib.base] /dataset/wathermalsprings render time 0.057 seconds [Fri May 15 14:07:59 2015] [error] 2015-05-15 14:07:59,176 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Fri May 15 14:08:03 2015] [error] 2015-05-15 14:08:03,147 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Fri May 15 14:13:21 2015] [error] [client 146.76.12.60] Error - <type 'exceptions.ValueError'>: invalid literal for int() with base 10: '' [Fri May 15 14:13:21 2015] [error] 2015-05-15 14:13:21,305 INFO [ckan.lib.base] /dataset/warockchem render time 0.056 seconds [Fri May 15 14:13:21 2015] [error] 2015-05-15 14:13:21,542 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Fri May 15 14:13:28 2015] [error] 2015-05-15 14:13:28,889 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Fri May 15 14:13:32 2015] [error] [client 146.76.12.60] Error - <type 'exceptions.ValueError'>: invalid literal for int() with base 10: '' [Fri May 15 14:13:32 2015] [error] 2015-05-15 14:13:32,123 INFO [ckan.lib.base] /dataset/wathermalsprings render time 0.054 seconds [Fri May 15 14:13:32 2015] [error] 2015-05-15 14:13:32,472 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Fri May 15 14:13:36 2015] [error] 2015-05-15 14:13:36,466 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Fri May 15 18:10:50 2015] [error] 2015-05-15 18:10:50,373 INFO [ckan.lib.base] /phpmyadmin/scripts/setup.php render time 0.003 seconds [Fri May 15 18:10:50 2015] [error] 2015-05-15 18:10:50,403 INFO [ckan.lib.base] /phpmyadmin/scripts/setup.php render time 0.029 seconds [Fri May 15 19:25:30 2015] [error] 2015-05-15 19:25:30,289 INFO [ckan.lib.base] /user/_logout render time 0.005 seconds [Fri May 15 19:25:30 2015] [error] 2015-05-15 19:25:30,547 INFO [ckan.lib.base] /user/logged_out render time 0.006 seconds [Fri May 15 19:25:30 2015] [error] 2015-05-15 19:25:30,745 INFO [ckan.lib.base] /user/logged_out_redirect render time 0.033 seconds [Fri May 15 19:25:31 2015] [error] 2015-05-15 19:25:31,251 INFO [ckan.lib.base] /api/i18n/en render time 0.001 seconds [Fri May 15 20:24:21 2015] [error] 2015-05-15 20:24:21,372 INFO [ckan.lib.base] / render time 0.065 seconds [Fri May 15 21:00:32 2015] [error] 2015-05-15 21:00:32,946 INFO [ckan.lib.base] / render time 0.097 seconds [Fri May 15 23:01:29 2015] [error] 2015-05-15 23:01:29,686 INFO [ckan.lib.base] /Panel/includes/templates/error.tpl render time 0.003 seconds [Fri May 15 23:01:29 2015] [error] 2015-05-15 23:01:29,716 INFO [ckan.lib.base] /Panel/includes/templates/error.tpl render time 0.030 seconds [Sat May 16 00:01:56 2015] [error] 2015-05-16 00:01:56,697 INFO [ckan.lib.base] / render time 0.064 seconds [Sat May 16 03:41:58 2015] [error] 2015-05-16 03:41:58,999 INFO [ckan.lib.base] / render time 0.067 seconds [Sat May 16 10:24:09 2015] [error] 2015-05-16 10:24:09,082 INFO [ckan.lib.base] / render time 0.068 seconds [Sat May 16 18:34:30 2015] [error] 2015-05-16 18:34:30,352 INFO [ckan.lib.base] / render time 0.067 seconds [Sat May 16 21:22:43 2015] [error] 2015-05-16 21:22:43,802 INFO [ckan.lib.base] /manager/html render time 0.003 seconds [Sat May 16 21:22:43 2015] [error] 2015-05-16 21:22:43,833 INFO [ckan.lib.base] /manager/html render time 0.030 seconds [Sun May 17 02:17:22 2015] [error] 2015-05-17 02:17:22,177 INFO [ckan.lib.base] / render time 0.066 seconds [Sun May 17 03:10:01 2015] [error] Exception AttributeError: "'NoneType' object has no attribute 'clearing'" in <bound method PluginEnvironment.del of Services for Environment '' [Sun May 17 03:10:01 2015] [error] > ignored [Sun May 17 03:10:01 2015] [error] Exception TypeError: "'NoneType' object is not callable" in <bound method LGEOS330.del of <shapely.geos.LGEOS330 object at 0x7f8bfb1f5418>> ignored [Sun May 17 03:10:01 2015] [error] Exception AttributeError: "'NoneType' object has no attribute 'clearing'" in <bound method PluginEnvironment.del of Services for Environment '' [Sun May 17 03:10:01 2015] [error] > ignored [Sun May 17 03:10:01 2015] [error] Exception TypeError: "'NoneType' object is not callable" in <bound method LGEOS330.del of <shapely.geos.LGEOS330 object at 0x7f8bfb1f6418>> ignored (END)

ccaudill commented 9 years ago

The AMI has been recreated. User will search for “CentOS 6.4 (x86_64) - Release Media” instead of "NGDS". The installation guide in the ReadMe of this GitHub repo has been updated. Also, t2.medium has been deprecated, so m3.medium must now be used.

ccaudill commented 9 years ago

From user Dave:

Dan, When I follow the instructions at the link you provided I get to step 21 just fine. Step 22 does not work (see below).

You email seems to indicate that I should follow the installation instructions for deploying on my own server. I tried this and got a number of errors (see farther below):

  1. Note the IP address in the Public IP column for this new instance. Once the launch has completed (it will take several minutes) you will be able to see your node at this IP address in any web browser.
  2. Important! Make sure these steps are completed immediately after launch. o At the NGDS landing page click Sign In. o Login with the default sysadmin account. The user is admin and the password is admin. o In the user settings change the password to something more secure.

login as: root Authenticating with public key "imported-openssh-key" Last login: Tue Jun 2 17:58:54 2015 from 65.242.97.254 [root@ip-172-31-63-147 ~]# rpm-qa | greep ckan -bash: greep: command not found -bash: rpm-qa: command not found [root@ip-172-31-63-147 ~]# rpm -qa | grep ckan [root@ip-172-31-63-147 ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP ql en 1000 link/ether 06:7e:ec:5f:5c:01 brd ff:ff:ff:ff:ff:ff inet 172.31.63.147/20 brd 172.31.63.255 scope global eth0 inet6 fe80::47e:ecff:fe5f:5c01/64 scope link valid_lft forever preferred_lft forever [root@ip-172-31-63-147 ~]# yum -y ca-certificates Loaded plugins: fastestmirror, presto No such command: ca-certificates. Please use /usr/bin/yum --help [root@ip-172-31-63-147 ~]# cd /etc/yum.repos.d/ [root@ip-172-31-63-147 yum.repos.d]# pwd /etc/yum.repos.d [root@ip-172-31-63-147 yum.repos.d]# dir CentOS-Base.repo CentOS-Debuginfo.repo CentOS-Media.repo CentOS-Vault.repo [root@ip-172-31-63-147 yum.repos.d]# curl -fsL0S http://packages.reisys.com/ckan/ngds/libxml2.repo [libxml2] name=NGDS libxml2 baseurl=http://packages.reisys.com/ckan/ngds/libxml2-repo/ enabled=1 gpgcheck=0 protect=0 [root@ip-172-31-63-147 yum.repos.d]# curl -fsL0S http://packages.reisys.com/ckan/ngds/ngds.repo [ckan] name=NGDS ckan baseurl=http://packages.reisys.com/ckan/ngds/ngds-repo/ enabled=1 gpgcheck=0 protect=0 [root@ip-172-31-63-147 yum.repos.d]# dir CentOS-Base.repo CentOS-Debuginfo.repo CentOS-Media.repo CentOS-Vault.repo [root@ip-172-31-63-147 yum.repos.d]# rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm Retrieving http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm warning: /var/tmp/rpm-tmp.sItUhR: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY Preparing... ########################################### [100%] 1:epel-release ########################################### [100%] [root@ip-172-31-63-147 yum.repos.d]# rpm -Uvh http://yum.postgresql.org/9.1/redhat/rhel-6.3-x86_64/pgdg-centos91-9.1-4.noarch.rpm Retrieving http://yum.postgresql.org/9.1/redhat/rhel-6.3-x86_64/pgdg-centos91-9.1-4.noarch.rpm warning: /var/tmp/rpm-tmp.dhZxwO: Header V4 DSA/SHA1 Signature, key ID 442df0f8: NOKEY Preparing... ########################################### [100%] 1:pgdg-centos91 ########################################### [100%] [root@ip-172-31-63-147 yum.repos.d]# rpm -Uvh http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-centos93-9.3-1.noarch.rpm Retrieving http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-centos93-9.3-1.noarch.rpm Preparing... ########################################### [100%] 1:pgdg-centos93 ########################################### [100%] [root@ip-172-31-63-147 yum.repos.d]# yum install -y libxml2-2.9.0 gdal-1.9.2 gdal-python-1.9.2 gdal-devel-1.9.2 yum-utils Loaded plugins: fastestmirror, presto Determining fastest mirrors Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again [root@ip-172-31-63-147 yum.repos.d]# curl -fsLOS http://packages.reisys.com/ckan/ngds/libxml2.repo [root@ip-172-31-63-147 yum.repos.d]# curl -fsLOS http://packages.reisys.com/ckan/ngds/ngds.repo [root@ip-172-31-63-147 yum.repos.d]# ^C [root@ip-172-31-63-147 yum.repos.d]# rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm Retrieving http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm warning: /var/tmp/rpm-tmp.DZR7Xo: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY Preparing... ########################################### [100%] package epel-release-6-8.noarch is already installed [root@ip-172-31-63-147 yum.repos.d]# [root@ip-172-31-63-147 yum.repos.d]# rpm -Uvh http://yum.postgresql.org/9.1/redhat/rhel-6.3-x86_64/pgdg-centos91-9.1-4.noarch.rpm Retrieving http://yum.postgresql.org/9.1/redhat/rhel-6.3-x86_64/pgdg-centos91-9.1-4.noarch.rpm Preparing... ########################################### [100%] package pgdg-centos91-9.1-4.noarch is already installed [root@ip-172-31-63-147 yum.repos.d]# [root@ip-172-31-63-147 yum.repos.d]# rpm -Uvh http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-centos93-9.3-1.noarch.rpm Retrieving http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-centos93-9.3-1.noarch.rpm Preparing... ########################################### [100%] package pgdg-centos93-9.3-1.noarch is already installed [root@ip-172-31-63-147 yum.repos.d]# yum install -y libxml2-2.9.0 gdal-1.9.2 gdal-python-1.9.2 gdal-devel-1.9.2 yum-utils Loaded plugins: fastestmirror, presto Loading mirror speeds from cached hostfile Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again [root@ip-172-31-63-147 yum.repos.d]# yum-config-manager --disable pgdg93 -bash: yum-config-manager: command not found [root@ip-172-31-63-147 yum.repos.d]# yum install ngds.ckan Loaded plugins: fastestmirror, presto Loading mirror speeds from cached hostfile Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again [root@ip-172-31-63-147 yum.repos.d]#

dano-reisys commented 9 years ago

I had to run (for some reason) yum upgrade ca-certificates --disablerepo=epel in order to be able to install the rpm.

Node http://52.6.223.139/ should work now

smrgeoinfo commented 9 years ago

no longer relevant