kindredgroup / puppet-forge-server

Private Puppet forge server supports local files and both v1 and v3 API proxies
69 stars 44 forks source link

UI not working #32

Open jordiprats opened 8 years ago

jordiprats commented 8 years ago

I'm unable to find my private modules using the web UI:

neither searching for the specific module

Found 0 module matching 'epel'

nor using a wildcard:

Found 0 module matching '*'

but it is working using the API (puppet module search):

[root@puppet2 modules]# puppet module search epel --module_repository=http://localhost:80
Notice: Searching http://localhost:80 ...
NAME              DESCRIPTION                                                                    AUTHOR        KEYWORDS                              
Jordi Prats-epel  Installs EPEL for CentOS 5,6 and 7                                             @Jordi Prats  Jordi Prats epel     

this is my setup:

root@78a947606c79:/usr/local/src# ./puppet-forge-server/bin/puppet-forge-server -m /var/forge/modules/
[2015-11-05 14:31:53] INFO   +- Daemonizing: false
[2015-11-05 14:31:53] INFO   |- Port: 8080
[2015-11-05 14:31:53] INFO   |- Host: default
[2015-11-05 14:31:53] INFO   |- Pidfile: default
[2015-11-05 14:31:53] INFO   |- Server: default
[2015-11-05 14:31:53] INFO   `- Backends:
[2015-11-05 14:31:53] INFO      - #<PuppetForgeServer::Backends::Directory:0x00000002160290 @module_dir="/var/forge/modules/", @log=#<PuppetForgeServer::Logger:0x00000002183150 @loggers=[#<Logger:0x00000002182fe8 @progname=nil, @level=1, @default_formatter=#<Logger::Formatter:0x00000002182f70 @datetime_format=nil>, @formatter=#<Proc:0x00000002182cc8@/usr/local/src/puppet-forge-server/lib/puppet_forge_server/logger.rb:30>, @logdev=#<Logger::LogDevice:0x00000002182ed0 @shift_size=nil, @shift_age=nil, @filename=nil, @dev=#<IO:<STDERR>>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x00000002182e80 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x00000002182db8>>>>]>, @readonly=false>
[2015-11-05 14:31:53] INFO  WEBrick 1.3.1
[2015-11-05 14:31:53] INFO  ruby 1.9.3 (2013-11-22) [x86_64-linux]
[2015-11-05 14:31:53] WARN  TCPServer Error: Address already in use - bind(2)
[2015-11-05 14:31:53] INFO  WEBrick::HTTPServer#start: pid=618 port=8080
WARN: tilt autoloading 'tilt/haml' in a non thread-safe way; explicit require 'tilt/haml' suggested.
80.169.245.33 - - [05/Nov/2015 14:31:55] "GET /modules?query=* HTTP/1.1" 200 2769 0.0230
80.169.245.33 - - [05/Nov/2015 14:31:55] "GET /modules?query=* HTTP/1.1" 200 2769 0.0232
80.169.245.33 - - [05/Nov/2015:14:31:55 CET] "GET /modules?query=* HTTP/1.1" 200 2769
http://puppet2/modules?query=epel -> /modules?query=*
80.169.245.33 - - [05/Nov/2015 14:31:55] "GET /css/puppetlabs.css HTTP/1.1" 304 - 0.0005
80.169.245.33 - - [05/Nov/2015 14:31:55] "GET /css/puppetlabs.css HTTP/1.1" 304 - 0.0007
[2015-11-05 14:31:55] WARN  Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
80.169.245.33 - - [05/Nov/2015:14:31:55 CET] "GET /css/puppetlabs.css HTTP/1.1" 304 0
http://puppet2/modules?query=* -> /css/puppetlabs.css
80.169.245.33 - - [05/Nov/2015 14:31:55] "GET /js/internal.js HTTP/1.1" 304 - 0.0003
80.169.245.33 - - [05/Nov/2015 14:31:55] "GET /js/internal.js HTTP/1.1" 304 - 0.0005
[2015-11-05 14:31:55] WARN  Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
80.169.245.33 - - [05/Nov/2015:14:31:55 CET] "GET /js/internal.js HTTP/1.1" 304 0
http://puppet2/modules?query=* -> /js/internal.js
80.169.245.33 - - [05/Nov/2015 14:31:56] "GET /img/forge-logo.png HTTP/1.1" 304 - 0.0004
80.169.245.33 - - [05/Nov/2015 14:31:56] "GET /img/forge-logo.png HTTP/1.1" 304 - 0.0006
[2015-11-05 14:31:56] WARN  Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
80.169.245.33 - - [05/Nov/2015:14:31:56 CET] "GET /img/forge-logo.png HTTP/1.1" 304 0
http://puppet2/modules?query=* -> /img/forge-logo.png
^C[2015-11-05 14:32:02] INFO  going to shutdown ...
[2015-11-05 14:32:02] INFO  WEBrick::HTTPServer#start done.
root@78a947606c79:/usr/local/src# find /var/forge/modules/
/var/forge/modules/
/var/forge/modules/eyp
/var/forge/modules/eyp/epel
/var/forge/modules/eyp/epel/eyp-epel-3.1.9.tar.gz
root@78a947606c79:/usr/local/src# tar tvf /var/forge/modules/eyp/epel/eyp-epel-3.1.9.tar.gz
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/
-rw-r--r-- root/root       242 2015-11-04 18:39 eyp-epel-3.1.9/Gemfile
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/spec/
-rw-r--r-- root/root       232 2015-11-04 18:39 eyp-epel-3.1.9/spec/spec_helper.rb.nuse
-rw-r--r-- root/root        52 2015-11-04 18:39 eyp-epel-3.1.9/spec/spec_helper.rb
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/spec/classes/
-rw-r--r-- root/root       140 2015-11-04 18:39 eyp-epel-3.1.9/spec/classes/init_spec.rb
-rw-r--r-- root/root       950 2015-11-04 18:39 eyp-epel-3.1.9/spec/classes/epel_spec.rb
-rw-r--r-- root/root       135 2015-11-04 18:39 eyp-epel-3.1.9/spec/classes/init_spec.rb.old
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/spec/fixtures/
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/spec/fixtures/modules/
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/spec/fixtures/modules/epel/
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/spec/fixtures/modules/epel/manifests/
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/spec/fixtures/manifests/
-rw-r--r-- root/root         0 2015-11-04 18:39 eyp-epel-3.1.9/spec/fixtures/manifests/site.pp
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/tests/
-rw-r--r-- root/root       506 2015-11-04 18:39 eyp-epel-3.1.9/tests/init.pp
-rw-r--r-- root/root      1260 2015-11-05 14:12 eyp-epel-3.1.9/metadata.json
-rw-r--r-- root/root       787 2015-11-05 14:12 eyp-epel-3.1.9/checksums.json
-rw-r--r-- root/root      2891 2015-11-04 18:39 eyp-epel-3.1.9/README.md
drwxr-xr-x root/root         0 2015-11-05 14:12 eyp-epel-3.1.9/manifests/
-rw-r--r-- root/root       726 2015-11-04 18:39 eyp-epel-3.1.9/manifests/init.pp
-rw-r--r-- root/root       834 2015-11-04 18:39 eyp-epel-3.1.9/manifests/params.pp
-rw-r--r-- root/root       633 2015-11-04 18:39 eyp-epel-3.1.9/Rakefile
root@78a947606c79:/usr/local/src# 

I've tried to install it using the gem and installing using git clone. Same result.

Is this a bug or am I missing something?

thanks,

i11 commented 8 years ago

Hi,

Yeah, that's weird. I'll come back to you after some more testing

jordiprats commented 8 years ago

Ok, thanks!

I'm using a docker container, this is my reciepe. Maybe there are some dependencies missing...

FROM ubuntu:14.04
MAINTAINER Jordi Prats

ENV HOME /root

#
# timezone and locale
#
RUN echo "Europe/Andorra" > /etc/timezone && \
    dpkg-reconfigure -f noninteractive tzdata

RUN export LANGUAGE=en_US.UTF-8 && \
    export LANG=en_US.UTF-8 && \
    export LC_ALL=en_US.UTF-8 && \
    locale-gen en_US.UTF-8 && \
    DEBIAN_FRONTEND=noninteractive dpkg-reconfigure locales

RUN DEBIAN_FRONTEND=noninteractive apt-get update

RUN DEBIAN_FRONTEND=noninteractive apt-get install wget -y

#ruby

RUN DEBIAN_FRONTEND=noninteractive apt-get install ruby gcc ruby-dev make autoconf -y

#https://github.com/unibet/puppet-forge-server

RUN gem install puppet-forge-server

RUN mkdir -p /var/forge/modules

VOLUME ["/var/forge/modules"]

EXPOSE 8080

CMD /usr/local/bin/puppet-forge-server -m /var/forge/modules -x http://forge.puppetlabs.com
i11 commented 8 years ago

Hi,

It took a while for me to find some time for this. It's rather challenging to reproduce issues with private modules. I've used your dockerfile and added a few of my private module files, but wasn't able to get the behaviour you've observed.

What's really odd is that puppet module search and the ui search is technically exactly the same operation, so I really don't see how it could differ without producing any errors in the log.

Could you please try testing the last commit https://github.com/unibet/puppet-forge-server/commit/45589187e669b45a04de29f7b03966a692ccb2b5 and let me know if it would be any different. Also if you could publish the content of your metadata.json (stripping whatever sensitive data you have), I could try building a mock module for testing.

i11 commented 8 years ago

UI and searching functionality has been addressed for the past several releases. Please let me know if this is still and issue. Otherwise due to inactivity I'll be closing it in a few days.

jordiprats commented 8 years ago

Sorry for my late response. Same result... Do I want me to install it using gem install or git clone?

We have released all our private modules on github, so you can take a look at their metadata.json.

In the example bellow I'm searching for this one: https://github.com/jordiprats/eyp-epel

But it's just the same for all of those: https://github.com/AtlasIT-AM

Access logs:

[root@puppet2 docker-privatepuppetforge]# [2016-01-25 13:17:58] INFO  Cache in RAM memory size: 1, MISS for url: http://puppet2/v3/modules?query=epel
80.169.245.33 - - [25/Jan/2016 13:17:58] "GET /modules?query=epel HTTP/1.1" 200 2772 0.0271
80.169.245.33 - - [25/Jan/2016 13:17:58] "GET /modules?query=epel HTTP/1.1" 200 2772 0.0273
80.169.245.33 - - [25/Jan/2016:13:17:58 CET] "GET /modules?query=epel HTTP/1.1" 200 2772
- -> /modules?query=epel
80.169.245.33 - - [25/Jan/2016 13:17:58] "GET /css/puppetlabs.css HTTP/1.1" 304 - 0.0006
80.169.245.33 - - [25/Jan/2016 13:17:58] "GET /css/puppetlabs.css HTTP/1.1" 304 - 0.0008
[2016-01-25 13:17:58] WARN  Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
80.169.245.33 - - [25/Jan/2016:13:17:58 CET] "GET /css/puppetlabs.css HTTP/1.1" 304 0
http://puppet2/modules?query=epel -> /css/puppetlabs.css
80.169.245.33 - - [25/Jan/2016 13:17:58] "GET /js/internal.js HTTP/1.1" 304 - 0.0005
80.169.245.33 - - [25/Jan/2016 13:17:58] "GET /js/internal.js HTTP/1.1" 304 - 0.0006
[2016-01-25 13:17:58] WARN  Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
80.169.245.33 - - [25/Jan/2016:13:17:58 CET] "GET /js/internal.js HTTP/1.1" 304 0
http://puppet2/modules?query=epel -> /js/internal.js
80.169.245.33 - - [25/Jan/2016 13:17:58] "GET /img/forge-logo.png HTTP/1.1" 304 - 0.0017
80.169.245.33 - - [25/Jan/2016 13:17:58] "GET /img/forge-logo.png HTTP/1.1" 304 - 0.0018
[2016-01-25 13:17:58] WARN  Could not determine content-length of response body. Set content-length of the response or set Response#chunked = true
80.169.245.33 - - [25/Jan/2016:13:17:58 CET] "GET /img/forge-logo.png HTTP/1.1" 304 0
http://puppet2/modules?query=epel -> /img/forge-logo.png

[root@puppet2 docker-privatepuppetforge]# 
[root@puppet2 docker-privatepuppetforge]# 
[root@puppet2 docker-privatepuppetforge]# puppet module search epel --module_repository=http://localhost:80
Notice: Searching http://localhost:80 ...
[2016-01-25 13:18:08] INFO  Cache in RAM memory size: 1, MISS for url: http://forge.puppetlabs.com/v3/modules?query=epel
172.17.42.1 - - [25/Jan/2016 13:18:09] "GET /v3/modules?query=epel HTTP/1.1" 200 33057 0.7685
172.17.42.1 - - [25/Jan/2016:13:18:08 CET] "GET /v3/modules?query=epel HTTP/1.1" 200 33057
- -> /v3/modules?query=epel
NAME                  DESCRIPTION                                                                AUTHOR          KEYWORDS                            
eyp-epel              Setup the EPEL package repo                                                @eyp            eyp epel                            
stahnma-epel          Setup the EPEL package repo                                                @stahnma        repository yum epel rpm             
zerlgi-epel           Setup the EPEL package repo                                                @zerlgi         redhat rhel linux epel              
flakrat-repo_epel     Configures the EPEL Yum repository config to point to a local mirror o...  @flakrat        repo module yum epel                
thias-rhel            Configure Red Hat Enterprise Linux specific files and services.            @thias          iptables yum firewall epel rpm      
stevenmerrill-carbon  Install Carbon from EPEL packages and configure it.                        @stevenmerrill  rhel epel centos carbon             
[root@puppet2 docker-privatepuppetforge]# 
dgvigil commented 8 years ago

I'm getting this same issue. Command line works without issue, but UI isn't showing any matches.

[2016-07-01 08:05:12] INFO  Using RAM memory LRUCache with time to live of 1800sec and max size of 250 elements
[2016-07-01 08:05:13] INFO   +- Daemonizing: true
[2016-07-01 08:05:13] INFO   |- Port: 9321
[2016-07-01 08:05:13] INFO   |- Host: default
[2016-07-01 08:05:13] INFO   |- Pidfile: /etc/puppetlabs/puppet-forge-server/server.pid
[2016-07-01 08:05:13] INFO   |- Server: default
[2016-07-01 08:05:13] INFO   `- Backends:
[2016-07-01 08:05:13] INFO      - #<PuppetForgeServer::Backends::Directory:0x000000015db3e8 @module_dir="/etc/puppetlabs/puppet-forge-server/modules", @log=#<PuppetForgeServer::Logger:0x00000001555540 @loggers=[#<Logger:0x00000001555478 @progname=nil, @level=1, @default_formatter=#<Logger::Formatter:0x00000001555450 @datetime_format=nil>, @formatter=#<Proc:0x00000001555298@/etc/puppetlabs/puppet-forge-server/lib/puppet_forge_server/logger.rb:31>, @logdev=#<Logger::LogDevice:0x00000001555400 @shift_size=1048576, @shift_age=0, @filename="/etc/puppetlabs/puppet-forge-server/log/server.log", @dev=#<File:/etc/puppetlabs/puppet-forge-server/log/server.log>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x000000015553d8 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x00000001555388>>>>]>, @readonly=false>
re.dac.ted.ip - - [01/Jul/2016 08:05:17] "GET / HTTP/1.1" 200 2606 0.0113
[2016-07-01 08:05:29] INFO  Cache in RAM memory size: 0, MISS for url: http://forge.REDACTED.com:9321/v3/modules?query=*
re.dac.ted.ip - - [01/Jul/2016 08:05:39] "GET /modules?query=* HTTP/1.1" 200 2661 10.0114

# puppet module search multipathd --module_repository=http://forge.REDACTED.com:9321
Notice: Searching http://forge.REDACTED.com:9321 ...
NAME             DESCRIPTION                          AUTHOR        KEYWORDS                                                                                       
9321-multipathd  Controls fiber services              @9321         9321 multipathd 

re.dac.ted.ip - - [01/Jul/2016 08:22:15] "GET /v3/modules?query=multipathd HTTP/1.1" 200 1098 0.0055
davidlawn commented 5 years ago

I've just had the same issue and it was a simple as the server hosting the UI not being able to resolve the URL it was hosting it on. Fixed the DNS and the search works.

Fabian1976 commented 4 years ago

I ran into the same issue and found that the UI stopped after 10 seconds and via the CLI it doesn't timeout. Looking through the code i found this line: https://github.com/kindredgroup/puppet-forge-server/blob/67d41449361810e2aa4bc5866effe76643ddfd5b/lib/puppet_forge_server/http/http_client.rb#L87

There is a hardcoded timeout of 10 seconds.