Convert To Spray

[reid-spencer]

[Migrated from Assembla ticket 21 - reported by @reid-spencer on 2014-10-11 23:57:01.000]

Decouple from Play 2.2 keeping only the good portions (config, json) and use sbt-web directly with spray. We don't need to expose routing or even much data from the web environment at all. Consequently just hand craft spray routes that correspond to the entities in Scrupal. Utilize sbt-twirl for templates. Incorporate the sbt-web asset pipeline. Decouple Scrupal's core from the vagaries of http as much as possible (i.e. it should be possible to add another transport for scrupal, such as akka or ACE).

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-10-17 03:37:49.000]

Integrate sbt-web - InProgress #21

• Upgrade build settings and webjar version for Play 2.3.4
• Setup the rjs/digest/gzip asset pipeline
• jshintify the javascript
• Resolve all paths to work with hew WebJars Asset controller
• Get pages displaying again
• Issues: still can't get QuoteCtrl working, marked element not rendering correctly

Committed to: scrupal:master Commit: [[r:5bd41cf955b8024fd19d23bb2f2f75d46b1421f4|scrupal:5bd41cf955]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-10-21 11:02:28.000]

Fix ng-show Problem - InProgress #21

• Replace global page_is_ready variable with Angular controller that has a page_is_ready scope variable
• Install the controller on the body element.

Committed to: scrupal:master Commit: [[r:fb1ac1c9271e4d74ff2a8991e159ee52bbfb4122|scrupal:fb1ac1c927]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-07 00:09:35.000]

Major Reorganiztion And Conversion To Spray - InProgress #21

This is a squash of 31 commits and results in a non-functional WIP that compiles and tests cleanly but is not a fully functional system because of disabled code. This is being committed for safe keeping and sharing. The changes are as follows:

Cleanup

• Move Patterns to utils
• Eliminate dead code

Build Fixes

• Remove cruft
• Make tests non-forking, non-parallel and unbuffered

Cleanup

• Attempt to get project build happening with more options
• Revise the debug script to add more options and support YourKit instrumentation.

Profiler

• Add a profiling utility for quick/focused profiling tasks

Improve Configuration

• Break out mongodb and akka configuration to separate files
• Add mongodb to logging configuration.

Simplify Debug Options

• Use sbt 0.13.6
• Add twril, native-packagers and spray's revolver sbt plugins
• Break project into sub-projects
• Break Build.scala into separate files
• Break dependence on most of Play
• Incorporate spray and akka
• Use Grizzled SLF4J
• Move many of the utilities to their own sub-project
• Compensate for lack of having Play
• Ensure all tests still work.
• Cleanup the code, remove Java heritage.
• Convert DataAccessObject to use BSONDocument and BSONCollection instead of JSON
• Create a Query Builder DSL in BSONValueBuilder
• Move package level things to package object
• Create initial test cases for BSONValueBuilder and DBContext
• Add IntelliJ settings to git
• Temporarily hide test directory because it doesn't build
• Make everything build with IntelliJ provided sbt tool
• Get everything compiling
• Fix spray libraries
• Fix .idea/modules
• Create scrupal-api and start moving code to it
• Make Registrable objects auto-register
• Fix Hasher tests
• Add Specs for Profiler and ScrupalComponent
• Make ConfigHelperSpec pass (resource path issue)
• Make all tests pass.
• Simplify the asT signature in Registrable
• Fix corresponding test cases.
• Make Registry log failures to unregister at finalize
• Avoid generating those logs in tests.
• Tone down logging of ignored exceptions
• Make DBContext work
• Fix bugs in emptyDatabase and isEmpty with "system.*" collection names
• Get the WithFakeDB utility working for test cases.
• DataAccessObject can now create, drop and insert/find arbitrary objects
• Redesign core classes for BSON storage with ReactiveMongo
• Get all core classes compiling.
• Add test for Icons
• Revise scrupal-core to separate out api from core
• Make it all work with MongoDB
• Convert types and objects to work with BSON not JSON
• Get it all working per the tests.
• Add ConfigKeys from old scrupal.api
• Better core.api package comments
• Reduce dependencies
• Remove .idea files for clean start
• Initial http module
• Get everything working according to tests
• Remove remaining stuff to web module
• Add IntelliJ 14 .idea files after import and build
• Get Feature And Site Directives Working
• Extract Validator from Type
• Forms: generic validation for BSONValue
• Start a scrupal-config project for the config wizard
• Factor AbstractRegistry out of Registry
• Make a "Utils Tests" test runner and share it
• Set the coding style
• Add Tests for each module
• Factor things out of objects and put them in Registry[T]
• Get run configurations for testing working properly.
• Update file colors to highlight modified files
• Permit higherKinds and existensials for scala compiler
• Revise codeStyle options
• Get Twirl compiler working through various build settings
• Implement some initial form support
• Add logger.xml to various projects
• Improve Form validation
• Comment out ConfigWizard -- later
• Exclude scrupal-web from the build, too many errors, too much redesign
• Make everything pass except ConfigWizard

Committed to: scrupal:master Commit: [[r:218de57ceb6b244b433cccb1ead019c5259ebd30|scrupal:218de57ceb]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-08 16:44:04.000]

Incorporate Basic Twirl Templates - InProgress #21

• Move the error pages and various page constructions from scrupal-web to scrupal-http
• Make a stubbed Assets controller that doesn't work but is sufficient for reverse routing assets
• Make the RouteConstructionFailure error use the new page infrastructure
• Make HasherSpec more tolerante of timing failures

Committed to: scrupal:master Commit: [[r:dfccf6a075829cafe53569baea8e6270fe09a48f|scrupal:dfccf6a075]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-15 04:19:42.000]

Project Configuration Update - InProgress #21

Committed to: scrupal:master Commit: [[r:3a01b6431a991f3f2c7ddfa1bcda565638cfb4e8|scrupal:3a01b6431a]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-15 04:20:35.000]

Project Configuration Update - InProgress #21

Committed to: scrupal:master Commit: [[r:dbf19889605c42b86d1d3a5e04df524b0cd048a2|scrupal:dbf1988960]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-15 04:29:24.000]

Minor Tweeks To Bootstrap - InProgress #21

• Provide a default implementation of bootstrap() method
• Remove redundant implementations of bootstrap()
• Make overrides as necessary for bootstrap()
• Use e.nonEmpty isntead of !e.isEmpty

Committed to: scrupal:master Commit: [[r:5aa40647bca5757204014113afa97bb6f3dcbe0b|scrupal:5aa40647bc]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-15 05:00:19.000]

Move Old Controllers To scrupal-http and Comment Out - InProgress #21

• Controllers will not be part of scrupal-web, that's for the web application
• Old controllers for assets, apidoc, home, etc. were written for play, comment them out
• We will eventually convert these to work with scrupal-http controllers
• Re-instate the scrupal-web module even though it is basically empty. It now compiles.
• Remove inspections in IDEA for Unit returning functions defined with =
• Make the top level project aggregate everything else so commands flow down from the top.

Committed to: scrupal:master Commit: [[r:0233aec3e512b968ee697a3c9dfc1902e940aa23|scrupal:0233aec3e5]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-18 05:20:29.000]

The Start Of An Asset Pipeline - InProgress #21

• Add SbtWeb, RequireJS, Gzip and Digest to the scrupal-http project
• Adjust build settings to get assets from .../assets directory (same level as src)
• Add a requirejs, digest, gzip pipeline
• Add a do-nothing js file, scrupal.js just to test the pipeline
• This is a step towards having the AssetsController actually find assets.

Committed to: scrupal:master Commit: [[r:a2fd93af33ccbd8c36908aa46ab2328180117df6|scrupal:a2fd93af33]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-18 06:14:44.000]

Get Build Working - InProgress #21

• Remove cruft from the modules.xml file
• The default build number is 0 not "-SNAPSHOT" and is required to be numeric now
• The -SNAPSHOT suffix is added to build number 0, but otherwise not.

Committed to: scrupal:master Commit: [[r:c585650d7a3738f80aaaa7107c2d65642e786be2|scrupal:c585650d7a]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-18 14:58:25.000]

Deal With Stream Marshalling - InProgress #21

• Provide an "asset_paths" configuration option read by AssetLocator
• The "asset_paths" provide additional paths to search for assets beyond resources
• Adjust AssetLocator.resourceOf to read from the "asset_paths"
• Add a StreamResult to wrap an InputStream as one way to return a result
• Make AssetLocator.fetch return a StreamResult instead of EnumeratorResult
• Endow Scrupal with a configurable timeout and a "withActorExec" to pass frequent implicits down
• Make AssetsController fetch the asset_paths from configuration, or empty
• Teach AssetsController just enough to serve a favicon
• Make the argument to Controller.routes have the Scrupal object be implicit
• Adjust EntityController to use the makeMarshallable functions in ScrupalMarshallers
• Enhance ScrupalMarshallers to deal with Stream, Exception, and Result[_] type results, and futures of them.

Committed to: scrupal:master Commit: [[r:a7c53f24c4cfbd958446ceefb8da7ffea78582cb|scrupal:a7c53f24c4]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-18 16:30:21.000]

Re-Enable Build / Add Test Case - InProgress #21

• Fix compilation problems in controllers
• Add AssetLocatorSpec to test the asset locator
• Add test data for AssetLocatorSpec to work with
• Make AssetLocator not require an ExecutionContext (no longer works with Enumerators)

Committed to: scrupal:master Commit: [[r:46e986ba0078187cdbaa80c59df9eb8ee0964ea0|scrupal:46e986ba00]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-18 20:07:10.000]

Get Stream Marshalling Working - InProgress #21

• Clearly, delegated marshallers need a content type.
• Also, one must actually invoke a marshaller to get data marshalled!
• Make StreamingResponseActor a top level class for easier Actor instantiation

Committed to: scrupal:master Commit: [[r:e9c0697ccc972992a728e262343776940cda02a2|scrupal:e9c0697ccc]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-18 20:23:27.000]

Switch to ContentType Instead Of MediaType - InProgress #21

• ContentType permits us to use alternative character encodings and is MediaType compatible.

Committed to: scrupal:master Commit: [[r:c74ed83430c09fbf50f5cfd9faf7667b48c66d9c|scrupal:c74ed83430]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-18 23:46:46.000]

Refactor Action/Result - InProgress #21

• Remove NodeProcessor as redundant, unused and illadvised
• Move EntityProcessor to core.api and renamed it ActionProcessor
• Revise EntityProcessor to process all Actions, not just Entity actions
• Rename TextResult as StringResult and use TxtResult for a Twirl TxtFormat type of result
• Move all the Result objects to their own file and document thoroughly
• Minor adjustments and clarifications in Disposition, added Unspecified as the first failure type
• Add two templates for converting exceptions to html and text, including stack trace
• Start work on a BSONDocument marshaller (low level binarizing needs work)
• Make the defaultLayout handle more of the new Result types
• Provide marshallers for OctetsResult and TxtResult
• Extend mystery_marshaller to decode all the Result types we have so far *

Committed to: scrupal:master Commit: [[r:cae353a96b310aaeb831d6774a3f1fdf5214c008|scrupal:cae353a96b]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-19 14:01:41.000]

Upgrade Asset Retrieval - InProgress #21

• Turn Akka logging to "INFO" level for better debugging
• Don't include "lib" in the webjars asset path
• Provide a PathOf.lib target for specifying webjar libraries
• Make a "head_route" Route in ScrupalService to provide any initial logic (e.g. debug logging)
• Pass the controllers generated by createControllers into createRouter instead of relying on Registry
• Make sure the AssetsController is always included and always highest priority
• Teach AssetsController about libraries, themes, and NotFound
• Add some RequestLogger utility functions for logging various things (like everything)
• Make Streaming of BSONValue a bit more type safe
• Move bootswatch themes into a assets/themes
• Make all asset folders regular and consistent and fully spelled out
• Make ErrorResult and ExceptionResult return the right HTTP StatusCodes, especially 404

Committed to: scrupal:master Commit: [[r:afddaa8e24ada6242bcf90360942835d93c9e4c2|scrupal:afddaa8e24]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-19 16:14:15.000]

Upgrade Asset Retrieval - InProgress #21

• Fix routes: never forget the ~ after each path!
• Move assets to scrupal-core and make it the place where assets are built
• Copy route information to new controllers from old play routes file
• Remove the old play routes file
• Revise copyright in a few files that still need updating

Committed to: scrupal:master Commit: [[r:a9d0823c20bb2825958f5b9a12687185bdc83faa|scrupal:a9d0823c20]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-19 20:56:48.000]

Get CSS Working - InProgress #21

• Integrate Less compilation into sbt-web
• Add web/less/main and web/gzipCompress directories to assets_paths
• Fix copyright settings in IDEA to use max length of 120 not 80
• Move marked and font-awesome dependencies to core
• Move requirejs dependences to web (http is now agnostic about content)
• Don't do rjs in core pipeline, it only delivers static pages, no javascript
• Add rjs to the scrupal-web pipeline
• Change default theme to bootswatch/cyborg
• get font-awesome as a library item
• Allow AssetLocator to opt out of finding minified resources (e.g. for icons and images)
• Treat themes separately in the AssetsController

Committed to: scrupal:master Commit: [[r:0db507ce12fd3da696f18a2726f8ce0925cb5fb1|scrupal:0db507ce12]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-20 04:56:48.000]

Integrate Welcome Site & EchoApp - InProgress #21

• Fix issues with enablement and type erasure. Have to do it filter style where the type information is known.
• Add a Template trait to allow storing content generating functions in the database (with arguments)
• Subclass Tamplate for TwirlTxt and TwirlHtml. Create BSONHandlers for new Template classes.
• Remove "actions" from the Entity type and invoke method/Action. This is what the facets are for
• Minor improvements to Layout to coincide with its sister, Template
• Remove the notion of a Default Site and create WelcomeSite. Instantiate it when no other site is loaded
• Move all the node types in CoreNodes into api.Node for collocation of companion object
• A Site's "host" field can be a regex. Allow for multiple matches. WelcomeSite Will match ".*" (anything)
• Complete the implementation of EchoEntity. All methods return results generated by Twirl Templates
• Make sure the Echo* things are enabled for each other at construction time, or else they get filtered out!
• Add templates for printing out BSONValue and friends
• Fix EntityController to get the plural/singular thing straight. Singular = facet

Committed to: scrupal:master Commit: [[r:3168d766fec772c8dd77ece324898eca72f439db|scrupal:3168d766fe]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-20 07:02:58.000]

Connect WelcomeController With Welcome Site - InProgress #21

• Various typos fixed and minor cleanups done.
• Get theming under control. Each site can specify a provider and name which is put into the context
• Allow Template objects to be Referenced from the database
• Factor SiteController out of EntityController to deal with site level things
• Make SiteController emit the site's root node for the / or empty path but otherwise pass to subclass
• Derive EntityController from SiteController and refactor
• Derive WelcomeController from SiteController and refactor
• Remove the old Welcome template as it is in core now
• Give all Registries a logger_identity that makes it clear they are a Registry

Committed to: scrupal:master Commit: [[r:a7a4bb26ac47b00b148403abc653e3455f6cf63e|scrupal:a7a4bb26ac]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-28 05:04:49.000]

Generalize Route Formation - InProgress #21

• Add a dispatch method to action for convenience that forwards to the Scrupal object
• Allow ActionProviders to have subordinate ActionProviders in a hierarchy.
• Make Entity, Application, Site, and Scrupal all be ActionProviders in that order of hierarchy
• Remove the Pathable trait and use the key in ActionProvider instead
• Define UserContext in terms ofr a Principal and make its applicator consistent with the others
• Make Entity a TerminalActionProvider that has no subordinates
• Adjust Entity interface to only require a Context not an ApplicationContext so it can appear in any Context.
• Replace the getAppEntities facility with processing the subordinate ActionProvider hierarchy
• Implement a context method in Controller that discerns a Scrupal context
• Create an ActionProviderController that generates the routes for an ActionProvider
• Define SiteController as an ActionProviderController for sites
• Define ApplicationController as an ActionProviderController for applications
• Rework EntityController to derive from ActionProviderController and override the action method
• Revise the WelcomeController to derive from ActionProviderController

Committed to: scrupal:master Commit: [[r:542fbd6b474ba01b8abb22115a7706ab538e6468|scrupal:542fbd6b47]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-11-28 14:38:26.000]

Update Project Files - InTest #21

• Changes from IDEA import.

Committed to: scrupal:master Commit: [[r:818dad05843089af34aec0ec8fc63577e60e9a97|scrupal:818dad0584]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-12-03 03:44:12.000]

Fix Test Case - InTest #21

• Make PrincipalSpec be repeatable by emptying database first.

Committed to: scrupal:master Commit: [[r:27d078af9046e309cea088f276def7d030ae647b|scrupal:27d078af90]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-12-04 04:34:09.000]

Various Rearrangements - InProgress #21

• Move scrupal-web to scrupal-opa
• Fix and move javascripts to scrupal-opa from main site
• Start work on an indexing feature for MarkedDoc
• Update copyright settings
• Include bootstrap javascript in main page *

Committed to: scrupal:master Commit: [[r:8e9bbc970d8de972609f3903881984a0708c579e|scrupal:8e9bbc970d]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2014-12-12 13:50:29.000]

Cleanup Some TODO Items - InTest #21

• RichResults no longer needed
• TODO items are out of date.

Committed to: scrupal:master Commit: [[r:b645cb5849156f1d894a53f5eba01bcdfea97b34|scrupal:b645cb5849]]

[reid-spencer]

[Migrated from comment on Assembla ticket 21 - comment by @reid-spencer on 2015-01-12 14:44:10.000]

This conversion is complete