search

coleslaw-org / coleslaw

Flexible Lisp Blogware
BSD 2-Clause "Simplified" License
560 stars 82 forks source link

old files keep lying around in staging-dir and output-dir #72

Open sternenseemann opened 9 years ago

sternenseemann commented 9 years ago

when not using the git backend.

kingcons commented 9 years ago

What do you mean by the "git backend"? More details on how to reproduce would be appreciated.

sternenseemann commented 9 years ago

.coleslawrc:

(:author "Lukas Epple"
 :deploy-dir "/home/lukas/Documents/blog/output/"
 :domain "http://lukasepple.de"
 :feeds ("lisp")
 :plugins ((static-pages))
 :routing ((:post           "posts/~a")
           (:tag-index      "tag/~a")
           (:month-index    "date/~a")
           (:numeric-index  "~d")
           (:feed           "~a.xml")
           (:tag-feed       "tag/~a.xml"))
 :sitenav ((:url "/" :name "Blog")
           (:url "/about.html" :name "About me")
           (:url "https://twitter.com/lukasepple" :name "Twitter")
           (:url "https://github.com/lukasepple" :name "Code")
           (:url "https://flickr.com/photos/sternenseemann" :name "Fotos")
           (:url "/imprint.html" :name "Imprint"))
 :staging-dir "/tmp/coleslaw/"
 :title "Lukas' blog"
 :theme "stars"
 :lang "de"
 :license "<a rel=\"license\" href=\"//creativecommons.org/licenses/by-sa/4.0/\"><img alt=\"Creative Commons License\" src=\"//i.creativecommons.org/l/by-sa/4.0/80x15.png\" /></a>")

After generating old generated files keep showing up in deploy-dir. That can be stopped by deleting staging-dir.

I myself fixed this like this: http://lukasepple.de/posts/coleslaw-helpers.html

sternenseemann commented 9 years ago

Just now an idea came to my mind. We could write a cli interface (using cl-launch) to solve that elegantely:

(Just like Hakyll)

sternenseemann commented 9 years ago

I think it would be nice if we agree on one set of cli command that gets implemented.

kingcons commented 9 years ago

I would love to see such a pull request, or at least more details on what a solution like that looks like. In fact, there's a note about looking into this in the HACKING docs here: https://github.com/redline6561/coleslaw/blob/master/docs/hacking.md#scripting-conveniences

I'm happy with your proposed set of commands. It might be nice for people who aren't doing "git-style" deploys to have a coleslaw post or coleslaw new command that takes a filename and creates it in the repo with the header filled out appropriately (i.e. date, format: md, title, empty tags field).

ghost commented 9 years ago

Good idea! Maybe we can also have a coleslaw serve command just like jekyll sreve to test site in local. Or coleslaw new a repo dir with shlyfile.lisp(just like caveman).I use shelly build and test my site now. Just put shlyfile.lisp in the repo dir with code 

(ql:quickload :hunchentoot)
(ql:quickload :coleslaw)

(defun build-blog ()
  (coleslaw:main #p"/home/qwfwq/blog/post/"))
(defun serve ()
  (hunchentoot:start (make-instance 'hunchentoot:easy-acceptor
                        :port 4242 :document-root #p"/home/qwfwq/blog/www/")))

Then I can use shly build-blog to build blog and use shly serve to test site in local.

(I am a common lisp newbie now，maybe I can do something some day.)

kingcons commented 9 years ago

@erlFelaP Good idea, thanks! I'd forgotten about shelly. That might be perfect for this.

PuercoPop commented 9 years ago

I currently use cl-launch to build, as the script is pretty much the same as the githook. The pros of cl-launch is that is looks like any other CLI tool from the perspective of the user, the cons are that they require an additional installation from the perspective of the user and that error messages are make clojure's stack traces look concise. a problem when not when debugging (as one can just use the REPL) but certainly for when an user wants to submit an error report. Maybe shelly fares better in this regard?

#!/usr/bin/cl -Q -sp coleslaw --entry entry
;;;;
;; This script assumes it is executed from the repository's top level directory
;; to determine correctly the blog-dir variable.
;;;;

(defun entry (argv)
  (declare (ignorable argv))

  (let 
      ((old-rev (inferior-shell:run/s
                 "git log --oneline -1 | awk -e '{print $1}'"))
       (blog-dir (uiop/os:getcwd)))
    (main blog-dir old-rev)))

If recently found about eazy-project which supports skeletons for scaffolding (akin to leiningen). Although it doesn't appear to be easily extendable. Should we open a branch to start working the command set proposed by Lucas + the serve commands?

kingcons commented 9 years ago

I would love that. I think shelly and cl-launch are both good options. I'd be happy to bundle code for both. Let's cut a branch called cli-commands.

sternenseemann commented 9 years ago

I would like to help doing that. As a downside of shelly you would not have a coleslaw binary, you'd have to invoke shelly rebuild which does not seem optimal to me.

kingcons commented 9 years ago

That's true. We could always have a shell alias alias coleslaw=shelly coleslaw or function to drop in your .profile. I agree it's not ideal and there may be better workarounds but I'll look into that later. If y'all just want to look into cl-launch for now I can have a look at shelly later on.

sternenseemann commented 9 years ago

Sounds good!

sternenseemann commented 9 years ago

I was able to create a simple coleslaw binary like this:

buildapp \
    --output coleslaw \
    --asdf-path "." \
    --load-system coleslaw \
    --asdf-tree "~/quicklisp/dists/quicklisp/software/" \
    --eval '(defun main (argv)
                (declare (ignore argv))
                (coleslaw:main "."))'

Biggest Problem: Size is 83MB.

Using buildapp would cause having a shell script which:

  1. Checks if quicklisp is available
  2. If true, calls sbcl to quickload everything needed (buildapp needs the asdf foo to be there before compiling)
  3. Call buildapp like above
  4. Else fails
sternenseemann commented 9 years ago

The buildapp binary is currently weirdly broken. A more sophisticated build script woud look like this:

#!/bin/sh
fatal() {
    echo "$1"
    exit 1
}

test -f ~/quicklisp/setup.lisp || fatal "You need quicklisp to use coleslaw"

sbcl --no-userinit --no-sysinit --non-interactive \
    --load ~/quicklisp/setup.lisp \
    --eval '(dolist (systems (quote ("closure-template" "3bmd" "3bmd-ext-code-blocks" "alexandria" "local-time"
                                     "inferior-shell" "cl-fad" "cl-ppcre" "closer-mop" "cl-unicode")))
                    (ql:quickload systems))' \
    --eval '(ql:write-asdf-manifest-file "quicklisp-manifest.txt")'

buildapp \
    --output coleslaw \
    --asdf-path "." \
    --load-system "coleslaw" \
    --manifest-file "quicklisp-manifest.txt" \
    --eval '(defun main (argv) 
                (coleslaw:main "."))' || fatal "buildapp failed!"

Running resulting binary fails for me with strange errors maybe someone can find out what they mean:

ASDF could not load sb-bsd-sockets because
Don't know how to REQUIRE sb-bsd-sockets.
See also:
  The SBCL Manual, Variable SB-EXT:*MODULE-PROVIDER-FUNCTIONS*
  The SBCL Manual, Function REQUIRE.

debugger invoked on a SB-INT:EXTENSION-FAILURE in thread
#<THREAD "main thread" RUNNING {10051D67A3}>:
  Don't know how to REQUIRE sb-bsd-sockets.
See also:
  The SBCL Manual, Variable SB-EXT:*MODULE-PROVIDER-FUNCTIONS*
  The SBCL Manual, Function REQUIRE

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RETRY                        ] Retry
                                     completing load for #<REQUIRE-SYSTEM "sb-bsd-sockets">.
  1: [ACCEPT                       ] Continue, treating
                                     completing load for #<REQUIRE-SYSTEM "sb-bsd-sockets">
                                     as having been successful.
  2: [TRY-RECOMPILING              ] Recompile impl and try loading it again
  3:                                 Retry
                                     loading FASL for #<CL-SOURCE-FILE "quicklisp" "impl">.
  4:                                 Continue, treating
                                     loading FASL for #<CL-SOURCE-FILE "quicklisp" "impl">
                                     as having been successful.
  5:                                 Retry ASDF operation.
  6: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the
                                     configuration.
  7: [RETRY                        ] Retry EVAL of current toplevel form.
  8: [CONTINUE                     ] Ignore error and continue loading file "/home/lukas/quicklisp/setup.lisp".
  9: [ABORT                        ] Abort loading file "/home/lukas/quicklisp/setup.lisp".
 10:                                 Retry EVAL of current toplevel form.
 11:                                 Ignore error and continue userinit file "/home/lukas/.sbclrc".
 12:                                 Abort userinit file "/home/lukas/.sbclrc".
 13:                                 Skip to toplevel READ/EVAL/PRINT loop.
 14: [EXIT                         ] Exit SBCL (calling #'EXIT, killing the process).

(SB-IMPL::REQUIRE-ERROR "Don't know how to ~S ~A." REQUIRE "sb-bsd-sockets")
0] ; 
; compilation unit aborted
;   caught 4 fatal ERROR conditions
*
sternenseemann commented 9 years ago

I think cl-launch could be a more reasonable alternative to buildapp

kingcons commented 9 years ago

Yeah, I wouldn't worry with standalone binaries/buildapp just yet. Some users would probably be fine with installing an 80mb executable to run coleslaw. We can look into it later. For now, let's just get a nice CLI interface up with cl-launch or shelly.

Trying to get a binary for folks that don't want to install SBCL, Quicklisp, etc would be very cool. I'll open an issue for that.

PuercoPop commented 9 years ago

I've pushed some preliminary changes to the cli-commands branch.

A couple of things:

Anyhow, thoughts/feedback?

Happy Holidays

kingcons commented 9 years ago

Awesome work, @PuercoPop! :thumbsup: I skimmed it and it looks good on the whole. I feel like process-parameters could be cleaner for some reason but haven't thought about it too carefully. Will try to test it out and report back in the next few days. :)

PuercoPop commented 9 years ago

I've pushed some more work to the cli-commands branch, rebasing to current master. I've switched from cl-launch to buildapp. Also using command-line-arguments to parse argv. however because I'm using a 'command' cli interface (a la git) we can't use the auto-generated usage. I've yet to evaluate if Clon handles the usage case better. Following pjb's writting style of using the documentation option of defpackage I've written the help text of the command in the defpackage form, I'm not convinced it is the best idea, maybe just using the docsting would be better? Also I've followed a one-package per-file (which I don't really like tbh because it makes working with the REPL a little bit harder) but could use the traditional one packages. file used in coleslaw. Anyhow below I list the items that need work before a merge, feel free to add what you think is missing and any feedback.

Things I want to do before merging

It also addresses #77.

kingcons commented 9 years ago

@PuercoPop: Quite excited to have a look this weekend. Thanks! :)

kingcons commented 9 years ago

@PuercoPop: Great job getting buildapp set up and making solid progress on the cli-commands branch. I definitely prefer the buildapp version to cl-launch. As noted, the config logic is a little messy. That probably just means it could use a rewrite anyway though. :)

The one other thing that bugs me is that the current plugin system interacts badly with binary deployment. In an ideal world, I'd love to be able to have a build farm kick out binary releases that users could just download, add configuration, and start blogging. Plugins must be compiled in at build time though. I'll think more on this but I we definitely should not block the initial release because of that.

PuercoPop commented 9 years ago

One way to handle plugins would be to make a proper asdf definition for each one. And an :after load-op method could then read a variable to figure which modules to load[1].

[1]: The variable would be cl-user for, as eclipse the wm, or a coleslaw.config package.