search

LD4P / sinopia_server

[Deprecated - switching to MongoDB] Sinopia Back-end CRUD Service. LDP-inspired, HTTP Server taking JSON-LD resources & administrative metadata.
Apache License 2.0
1 stars 1 forks source link

can we make the trellis server logs more accessible? #51

Closed jmartin-sul closed 5 years ago

jmartin-sul commented 5 years ago

currently, the trellis server logs are only accessible from inside the container running trellis on AWS. we can access the logs of the currently running container by SSHing into the instance, but if the container gets torn down, those logs are gone. they will become totally inaccessible if we run trellis as a fargate task (e.g. #50), in which case we'll have no access at all into the container running trellis, because fargate tasks are entirely managed by AWS.

a cursory web search indicates that at least a couple AWS-focused log4j appenders exist? are maybe even maintained?

jcoyne commented 5 years ago

@jmartin-sul if we run it in fargate, can we log to stdout (and thus Cloudwatch?)

jmartin-sul commented 5 years ago

@jcoyne possibly. i think @rsmith11 could answer that more definitively (the "can we log to stdout (and thus Cloudwatch?)" part).

but, iirc: it would be difficult to run it in fargate, because we need an EFS mount, which fargate doesn't support.

also, iirc: there is a more cloud-native version of trelis, which does not need the file system for memento storage. see: https://github.com/trellis-ldp/trellis-ext-aws

50

we just haven't had time to give it a try yet. but it would be nice to switch over to trellis-ext-aws, if possible.

jmartin-sul commented 5 years ago

copying helpful clarification from aaron coburn in the other issue:

Hi All, the big difference with the AWS-based Trellis is that it uses S3 for storing binaries and mementos (rather than a filesystem). It also ties in directly to SNS for notifications. Using trellis-ext-aws-rds would be most similar to what you're already using.

aaron-collier commented 5 years ago

There is currently some logging in Cloudwatch for trellis. Is it missing some actual logs that are only on the machine?

Standard practice with images is to log to stdout (as @jcoyne mentions) so they are picked up by cloudwatch for retention.

acoburn commented 5 years ago

Trellis generates two kinds of logs: request logs (i.e. GET /path/to/resource 12:00:00 GMT) and application logs (any stack traces and/or application-level logging). Both of these are controlled by configuration.

The current configuration for sinopia sends request logs to a file and sends application logs to both a file and to the console.

If you would like to send request logs to the console, you can simply add an appender of type: console to the configuration here: https://github.com/LD4P/sinopia_server/blob/master/trellis/etc/config.yml#L9

Hope that helps.

jmartin-sul commented 5 years ago

@rsmith11 confirmed that our ECS instance will send stdout to cloudwatch (i.e., that capability isn't exclusive to fargate deployments).

i've put up a PR to send request logs to stdout (as @acoburn points out, we're already doing that for application logs).

will close this issue once: