ZYLYTY

[ZYLYTY]

Hello @cpfergus1,

Thank you for submitting your challenge to us.

You are receiving this message because we found the score of your project quite low, which usually triggers some alerts in our system that highlight the possibility of your submission having been unfairly scored by our system.

I will be putting here some findings so we can try to reach a fairer score. 😃

[ZYLYTY]

We started by assessing your ENVs, to make sure they were being injected to your runtime container, but we found an issue:

As you can see the port to listen is set to 4000, and your app is being able to get it perfectly as you can see here:

Although you can perfectly see the ENV, for some reason your server is still assuming port 3000.

[ZYLYTY]

Another issue we found is that your server is creating two additional databases:

But the objective is to just use the one provided to you via the env DB_SCHEMA_NAME, otherwise the tests will fail.

[ZYLYTY]

The registration endpoint is failing with a basic request:

$ curl -X POST "http://localhost:8080/user/register"      -H "Content-Type: application/json"      -d '{
           "username": "fakeuser123",
           "password": "yourSecurePassword",
           "email": "fakeuser123@example.com"
         }'

Both this endpoint and the login one are critical for everything else to be tested properly. This is probably the major reason why your functionality tests all failed.

[ZYLYTY]

Please let us know when you fix these issues, so we can re-fetch your submission for evaluation.

Thanks.

[cpfergus1]

Thanks for reaching out!

I have corrected the server errors with #91e19e and corrected the user controller errors by removing a before hook which was left in after attempting to clean my commits 😓

The server errors were caused by overwriting my binding in puma config when executing rails in Dockerfile and two database schemas were being created because in the development environment that is default.

It looks to be working now I and accept my fate on the eval 😄

[ZYLYTY]

Hello @cpfergus1 we are witnessing these error logs:

ruby_1        | rails aborted!
ruby_1        | ArgumentError: Missing `secret_key_base` for 'production' environment, set this string with `bin/rails credentials:edit` (ArgumentError)
ruby_1        | 
ruby_1        |         raise ArgumentError, "Missing `secret_key_base` for '#{Rails.env}' environment, set this string with `bin/rails credentials:edit`"
ruby_1        |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ruby_1        | /app/config/environment.rb:5:in `<main>'
ruby_1        | Tasks: TOP => db:prepare => db:load_config => environment
ruby_1        | (See full trace by running task with --trace)
zylytypractice_ruby_1 exited with code 1

Maybe some config is missing on rails?

[cpfergus1]

Yeah, I put up another commit that should fix that but I want to test on another machine before asking you to check again, I will reply shortly

On Mar 28, 2024, at 8:51 AM, ZYLYTY @.***> wrote:

Hello @cpfergus1 https://github.com/cpfergus1 we are witnessing these error logs:

ruby_1 | rails aborted! ruby_1 | ArgumentError: Missing secret_key_base for 'production' environment, set this string with bin/rails credentials:edit (ArgumentError) ruby_1 | ruby_1 | raise ArgumentError, "Missing secret_key_base for '#{Rails.env}' environment, set this string with bin/rails credentials:edit" ruby_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ruby_1 | /app/config/environment.rb:5:in `

' ruby_1 | Tasks: TOP => db:prepare => db:load_config => environment ruby_1 | (See full trace by running task with --trace) zylytypractice_ruby_1 exited with code 1 Maybe some config is missing on rails?

— Reply to this email directly, view it on GitHub https://github.com/cpfergus1/zylytyPractice/issues/1#issuecomment-2025419813, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQICOBTQYB5OKAFGNMTIVA3Y2QN6ZAVCNFSM6AAAAABFMLPUZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRVGQYTSOBRGM. You are receiving this because you were mentioned.

[ZYLYTY]

Try running directly with the docker container, which is the closest to the environment where it will be tested in ZYLYTY.

[cpfergus1]

Try running directly with the docker container, which is the closest to the environment where it will be tested in ZYLYTY.

My containers on my machine are green, I am not 100% sure why you are getting this error and I think I fixed it, but I want to test it on another machine before you do 😄

[ZYLYTY]

Can you try running this docker-compose.yaml in the root of your project?

version: '3.8'

services:
  ruby:
    build: .
    ports:
      - "8080:4000"
    environment:
      DB_USERNAME: postgres
      DB_PASSWORD: test
      DB_SCHEMA_NAME: main_db
      DB_PORT: 5432
      DB_HOST: postgresdb  # Use the service name as the host within Docker network
      ADMIN_API_KEY: DEFAULT-API-KEY
      API_LISTENING_PORT: 4000
    depends_on:
      - postgresdb
    networks:
      - app-network

  postgresdb:
    image: postgres:15
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: test
      POSTGRES_DB: main_db
    ports:
      - "5432:5432"  # Optional: Expose PostgreSQL port on the host for direct access
    networks:
      - app-network

networks:
  app-network:
    driver: bridge

With: docker-compose down && docker-compose build && docker-compose up

[cpfergus1]

All good on this side

I am almost done setting up another system to test 😓

[ZYLYTY]

Such an odd error.

If we have the same codebase, the container is also the same... Our systems run Linux Debian, not Docker Desktop, although the odds of this being the culprit are low.

I am on commit 5e1117a43eeeded915bbc297a54b7ae4e654bed0. My reproduction steps are:

1. Cloning your repo, on main (which is the 5e1117a43eeeded915bbc297a54b7ae4e654bed0 commit)
2. Creating that docker-compose.yaml in the root of your project
3. Running docker-compose down && docker-compose build && docker-compose up

And the result:

Attaching to zylytypractice_postgresdb_1, zylytypractice_ruby_1
postgresdb_1  | The files belonging to this database system will be owned by user "postgres".
postgresdb_1  | This user must also own the server process.
postgresdb_1  | 
postgresdb_1  | The database cluster will be initialized with locale "en_US.utf8".
postgresdb_1  | The default database encoding has accordingly been set to "UTF8".
postgresdb_1  | The default text search configuration will be set to "english".
postgresdb_1  | 
postgresdb_1  | Data page checksums are disabled.
postgresdb_1  | 
postgresdb_1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgresdb_1  | creating subdirectories ... ok
ruby_1        | Waiting for the database to be ready...
postgresdb_1  | selecting dynamic shared memory implementation ... posix
postgresdb_1  | selecting default max_connections ... 100
postgresdb_1  | selecting default shared_buffers ... 128MB
postgresdb_1  | selecting default time zone ... Etc/UTC
postgresdb_1  | creating configuration files ... ok
postgresdb_1  | running bootstrap script ... ok
postgresdb_1  | performing post-bootstrap initialization ... ok
postgresdb_1  | syncing data to disk ... ok
postgresdb_1  | 
postgresdb_1  | 
postgresdb_1  | Success. You can now start the database server using:
postgresdb_1  | 
postgresdb_1  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
postgresdb_1  | 
postgresdb_1  | initdb: warning: enabling "trust" authentication for local connections
postgresdb_1  | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
postgresdb_1  | waiting for server to start....2024-03-28 17:35:25.345 UTC [48] LOG:  starting PostgreSQL 15.6 (Debian 15.6-1.pgdg120+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
postgresdb_1  | 2024-03-28 17:35:25.348 UTC [48] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgresdb_1  | 2024-03-28 17:35:25.357 UTC [51] LOG:  database system was shut down at 2024-03-28 17:35:25 UTC
postgresdb_1  | 2024-03-28 17:35:25.363 UTC [48] LOG:  database system is ready to accept connections
postgresdb_1  |  done
postgresdb_1  | server started
postgresdb_1  | CREATE DATABASE
postgresdb_1  | 
postgresdb_1  | 
postgresdb_1  | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgresdb_1  | 
postgresdb_1  | waiting for server to shut down....2024-03-28 17:35:25.516 UTC [48] LOG:  received fast shutdown request
postgresdb_1  | 2024-03-28 17:35:25.519 UTC [48] LOG:  aborting any active transactions
postgresdb_1  | 2024-03-28 17:35:25.520 UTC [48] LOG:  background worker "logical replication launcher" (PID 54) exited with exit code 1
postgresdb_1  | 2024-03-28 17:35:25.520 UTC [49] LOG:  shutting down
postgresdb_1  | 2024-03-28 17:35:25.523 UTC [49] LOG:  checkpoint starting: shutdown immediate
postgresdb_1  | 2024-03-28 17:35:25.570 UTC [49] LOG:  checkpoint complete: wrote 918 buffers (5.6%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.014 s, sync=0.024 s, total=0.050 s; sync files=301, longest=0.004 s, average=0.001 s; distance=4223 kB, estimate=4223 kB
postgresdb_1  | 2024-03-28 17:35:25.574 UTC [48] LOG:  database system is shut down
postgresdb_1  |  done
postgresdb_1  | server stopped
postgresdb_1  | 
postgresdb_1  | PostgreSQL init process complete; ready for start up.
postgresdb_1  | 
postgresdb_1  | 2024-03-28 17:35:25.636 UTC [1] LOG:  starting PostgreSQL 15.6 (Debian 15.6-1.pgdg120+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
postgresdb_1  | 2024-03-28 17:35:25.637 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
postgresdb_1  | 2024-03-28 17:35:25.638 UTC [1] LOG:  could not create IPv6 socket for address "::": Address family not supported by protocol
postgresdb_1  | 2024-03-28 17:35:25.644 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgresdb_1  | 2024-03-28 17:35:25.651 UTC [64] LOG:  database system was shut down at 2024-03-28 17:35:25 UTC
postgresdb_1  | 2024-03-28 17:35:25.656 UTC [1] LOG:  database system is ready to accept connections
postgresdb_1  | 2024-03-28 17:35:28.083 UTC [69] FATAL:  database "main" does not exist
ruby_1        | rails aborted!
ruby_1        | ArgumentError: Missing `secret_key_base` for 'production' environment, set this string with `bin/rails credentials:edit` (ArgumentError)
ruby_1        | 
ruby_1        |         raise ArgumentError, "Missing `secret_key_base` for '#{Rails.env}' environment, set this string with `bin/rails credentials:edit`"
ruby_1        |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ruby_1        | /app/config/environment.rb:5:in `<main>'
ruby_1        | Tasks: TOP => db:prepare => db:load_config => environment
ruby_1        | (See full trace by running task with --trace)
zylytypractice_ruby_1 exited with code 1
postgresdb_1  | 2024-03-28 17:35:38.094 UTC [70] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:35:48.114 UTC [71] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:35:58.130 UTC [73] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:36:08.142 UTC [74] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:36:18.159 UTC [75] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:36:28.171 UTC [77] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:36:38.182 UTC [78] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:36:48.200 UTC [79] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:36:58.210 UTC [81] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:37:08.221 UTC [82] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:37:18.232 UTC [83] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:37:28.242 UTC [85] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:37:38.260 UTC [86] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:37:48.271 UTC [87] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:37:58.281 UTC [89] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:38:08.298 UTC [90] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:38:18.318 UTC [91] FATAL:  database "main" does not exist
postgresdb_1  | 2024-03-28 17:38:28.328 UTC [93] FATAL:  database "main" does not exist

[ZYLYTY]

Found the issue:

How are you setting this ENV in your container? Our system does not inject this env.

[cpfergus1]

That is strange, I was thinking I may have to set that based on what I was reading but my credentials.yml is not setup to receive that ENV variable... Rails must be handling that along with the fact I have the master.key on my system. Unfortunately they also comment that adding the SECRET_KEY_BASE to a docker file is wrong because it is exposed 😆

I just reverted changes to run on the development environment (and ensure the the database is read correctly) to bypass this issue for this exercise, but if by injecting the secret_key_base worked on your end, I guess there is no reason to redownload

[ZYLYTY]

That is strange, I was thinking I may have to set that based on what I was reading but my credentials is not setup to receive that ENV variable... Rails must be handling that. Unfortunately they also comment that adding the SECRET_KEY_BASE to a docker file is wrong because it is exposed 😆

For the purpose of this test maybe you can inject it from the entrypoint.sh, this way it is not directly expressed in the Dockerfile.

But yes, ideally in production this ENV would come from some kind of vault of the surrounding infrastructure.

[ZYLYTY]

but if by injecting the secret_key_base worked on your end, I guess there is no reason to redownload

I was able to inject it manually, but our evaluation system won't, so this information must come with the container. The only ENVs it injects are the ones in that table from the instructions.

[cpfergus1]

I will update accordingly

[cpfergus1]

Updated!

Please let me know if that did it 😅

[cpfergus1]

I just found the answer. My credentials yml won't be read without that master.key. I am assuming my jwt_secret will also break now... That is my oversight

[ZYLYTY]

Anything that requires ENVs which are not from that table will break yes.

Let us know when we can re-evaluate.

[ZYLYTY]

For these extra envs, maybe you can make the project load them from the .env file if they are not provided by the system at runtime.

[cpfergus1]

For these extra envs, maybe you can make the project load them from the .env file if they are not provided by the system at runtime.

I love the suggestion, only issue I could see is that credentials would still live on the repository... In the spirit of the assignment without having those extra ENV variables and maintaining security, I will generate the ENV variables if they do not exist ignoring the fact that this is not suitable for real production environments 😄 . I pushed my changes and this should hopefully be the last!

[cpfergus1]

Is everything looking okay on your side for boot up?

[ZYLYTY]

Can you confirm in your cookie if you are setting the value session, I think you are setting it as sessionId instead?

[cpfergus1]

Yes I was setting the sessionId instead of session. I misunderstood that requirement, I updated the code to reflect the change so that session will be set instead of sessionId.

Thank you for your patience and notifying me of these issues!

[ZYLYTY]

We will be importing your challenge again somewhere until monday. Feel free to have a further look into the code 😉

[ZYLYTY]

Hello @cpfergus1 I re-run the evaluator on your project in debug mode so you can see that is failing in your tests.

These logs are still a beta feature so please let us know if you find any bug.

You can see the logs in the Details table in your home:

Ignore the rest of your score for now, while you have many of the functional tests failing.

[cpfergus1]

Now we are getting to the part where I missed some things in your story.... I used 'Authorization' instead of 'Token'.

[cpfergus1]

This is great learning lesson for me, thank you for your time. I felt like I could write pretty clean code, but it turns out I had a lot of difficulty following details the story 😓

[ZYLYTY]

That's good feedback. Do you think the instructions could have been written differently? What made you miss those small details?

You can still retry if you want. These are minor things. I would be more concerned on the endpoints returning HTTP 500, for example the login one returns 500 even when returning a successful Set-Cookie header.

[cpfergus1]

I think it was me just falling back on conventions which is more of an issue for myself. For me it was just, it was the thought process of this belongs here and not here! I should be able to take your spec and build it as required even if it doesn't fit the mold 😆

I agree on the 500, which is silly because nowhere in your story has a 500 error showing up, yet I still chose to have it for fallback to unknowns. That was deliberate and wrong choice that I made there.

I would still love to retry, because despite these smaller detail issues, I am still very interested in how your tests perform in my code. So I know where to focus my attention... I will definitely continue to use your platform in 3 months to test myself again.

[ZYLYTY]

That's an ongoing discussion within our team for months.

We try to avoid some conventions to force the candidate to adapt the project a bit and pay attention to some small details, however on the other hand the result is that you can end up compromising an entire score due to these small details, which may be unfair for the candidate.

Maybe in the future we will go for the conventions, at least in the more critical endpoints, so that small attention details don't compromise the entire solution.

PS. When I meant retry, I was referring to now, only if you still have time available ofc. If you make another commit that you think fixes these issues, I can re-run the evaluator. No need to wait 3 months to fix this. Specially because by then the challenge will be of a different topic.

Thanks for the feedback!

[cpfergus1]

That's an ongoing discussion within our team for months.

We try to avoid some conventions to force the candidate to adapt the project a bit and pay attention to some small details, however on the other hand the result is that you can end up compromising an entire score due to these small details, which may be unfair for the candidate.

Maybe in the future we will go for the conventions, at least in the more critical endpoints, so that small attention details don't compromise the entire solution.

PS. When I meant retry, I was referring to now, only if you still have time available ofc. If you make another commit that you think fixes these issues, I can re-run the evaluator.

Thanks for the feedback!

Absolutely let's try, I am around for bit!

I 100% get your strategy and perhaps it is something that could be integrated into the tests? Do you think it may be too difficult to automate that check to see whether the candidate utilized convention over the story variable? That way you could flag the specific failure over having the entire suite fail, but this would only make sense if you actually have more candidates failing this way

[ZYLYTY]

Yes, we might look for "Authorization" with and without the "Bearer" prefix. The same with the listening port (we set one random, but then fallback to 3000 in case for some reason the env fails).

Hopefully this will capture most cases.

[cpfergus1]

By the way, I did not specifically mention this, but I did make the appropriate change and pushed a commit to fix the header issue

[ZYLYTY]

@cpfergus1 I just tested it locally, I am getting 400 with the following payload:

Expected status code 200 (and a response with username: test and e-mail: test@zylyty.com), got 400 instead
Request history:
    Faulty Request:
        POST /user/login

        {"username":"test","password":"testpwd111!!!"}
    Response:
        HTTP Code: 400
        X-Frame-Options: SAMEORIGIN
        X-XSS-Protection: 0
        X-Content-Type-Options: nosniff
        X-Download-Options: noopen
        X-Permitted-Cross-Domain-Policies: none
        Referrer-Policy: strict-origin-when-cross-origin
        Set-Cookie: session=eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjozLCJleHAiOiIxNzExODE2MTIwIn0.xKrBnrAkIZuloOaiUGg2tRpr2FB81ixGXzlLhzHSpa8
        Content-Type: application/json; charset=utf-8
        Cache-Control: no-cache
        X-Request-Id: 9f20b55a-df0d-40da-bf93-e3c0b780953b
        X-Runtime: 0.353184
        vary: Origin
        Transfer-Encoding: chunked

The Set-Cookie is there, but HTTP code is 400.

Also, don't forget that there must always exist the "Default" category.

The GET /categories seems to be having issues as well.

[cpfergus1]

Silly question, but was the user registered?

I am seeing a 200 on my side after first registering the user:

[cpfergus1]

Also I am surprised that category default was not generated as rails db:prepare is supposed to run create:migrate:seed. I now ensured creation by adding rails db:seed specifically Are you clearing the DB in between tests? I did not account for that

☝️ That did not make sense 😆

[cpfergus1]

It looks like it set the cookie in your response which is interesting, because that occurs right after the user is logged in:

  def login
    @user = User.find_by(username: params[:username])
    if @user&.authenticate(params[:password])
      @token = issue_token(@user) 
      response.set_cookie('session', @token) <------ HERE
      render 'users/login', status: :ok
    else
      head :unauthorized
    end
  end

[cpfergus1]

I just tested an non existent user and I am getting a 401 which is expected for non registered users 🤔 :

[ZYLYTY]

The user seems to exist, and the session token you return also seems valid. The only thing off with the login endpoint seems to be the 400 code it returns upon the successful login 😆

[ZYLYTY]

Does your container produce logs on a 400 error? If so I can try to peek inside.

[cpfergus1]

We will not get a trace in production, that was a security measure but I can update it to get around that

should I move forward with that?

[cpfergus1]

Did you get an error in the body?

[cpfergus1]

  def truncated_error(error)
    error_info = {
      error: "Internal Server Error",
      exception: "#{error.class.name} : #{error.message}",
    }
    error_info[:trace] = error.backtrace if Rails.env.development?
    render json: error_info.to_json, status: :bad_request
  end

We might be able to see the error class

in exception

[ZYLYTY]

I just tried to get to the logs, you are printing everything to the console, that will put you at a slight disadvantage in terms of energy consumption. You could for eg. truncate all the payloads up to 1000 chars for example. We have some stress load tests that will fill up your logs with huge payloads, which can also contribute to request timeouts.

[ZYLYTY]

So, it seems the test user gets registered:

You can see here the failing login ones:

[cpfergus1]

I just tried to get to the logs, you are printing everything to the console, that will put you at a slight disadvantage in terms of energy consumption. You could for eg. truncate all the payloads up to 1000 chars for example. We have some stress load tests that will fill up your logs with huge payloads, which can also contribute to request timeouts.

That is a great suggestion! Is this test creating the user then trying to log them in along with a bunch of others at the same time?

[ZYLYTY]

I just tried to get to the logs, you are printing everything to the console, that will put you at a slight disadvantage in terms of energy consumption. You could for eg. truncate all the payloads up to 1000 chars for example. We have some stress load tests that will fill up your logs with huge payloads, which can also contribute to request timeouts.

That is a great suggestion! Is this test creating the user then trying to log them in along with a bunch of others at the same time?

Basically every test that requires authentication performs the login first.

But even the basic login test fails with a 400:

Maybe because the evaluator is not sending the Content-Type might be affecting this? But that's strange because in that case it would affect all the other endpoints as well.

[ZYLYTY]

Also, don't forget the payload that /login needs to return, it is not just the Set-Cookie header and HTTP 200, there's a JSON body it should return as well.