search

copperdogma / dnd-phb-5e-index

Dungeons & Dragons 5th Edition Revised Index
104 stars 11 forks source link

Make the HTML accessible online (potentially via Github pages) #8

Open ndguarino opened 8 years ago

ndguarino commented 8 years ago

It might be worth making the HTML version accessible without having to clone/download the raw file. Github Pages is actually perfect for this, and fairly simple to set up (https://help.github.com/articles/user-organization-and-project-pages/).

I'm more than happy to submit a pull request with the proper branch and an update to Readme.md if you think it's a good idea.

copperdogma commented 8 years ago

Do you mean a 5-column final output HTML version or an HTML version of the raw list?

ndguarino commented 8 years ago

The existing 5-column layout since it's the only HTML file existing at the moment in the repository, though truthfully both might be useful.

copperdogma commented 8 years ago

I actually started doing that, but HTML makes it pretty difficult to format a 5-column outline, and the hanging indents are REALLY hard if not impossible. (I've worked with HTML5/CSS for 5 years.)

So after that setback I thought "aren't most people going to view this on a computer or print it out and tuck it into the back of their PHBs?" so I decided just to format it in Word and output a PDF which would satisfy both of those use cases.

Can you think of something I missed where HTML would be a better format?

ndguarino commented 8 years ago

Search is the biggest usecase I can think of (and even if I'm in the minority, the search and html format would be the most useful features for me, especially if it can be mobilized). I noticed you have that as Issue #7, so I'll probably submit a pull request for that when I get a bit of spare time and can modify the stylesheet on one of my documentation layouts with those features. Probably later tonight or tomorrow.

copperdogma commented 8 years ago

Ah I see what you're getting at! I envisioned the searchable version to be more like a mobile-friendly single column affair with live results. But let's see what you come up with! On Sun, Nov 15, 2015 at 11:20 AM Falanyx notifications@github.com wrote:

Search is the biggest usecase I can think of (and even if I'm in the minority, the search and html format would be the most useful features for me, especially if it can be mobilized). I noticed you have that as Issue

7 https://github.com/copperdogma/dnd-phb-5e-index/issues/7, so I'll

probably submit a pull request for that when I get a bit of spare time and can modify the stylesheet on one of my documentation layouts with those features. Probably later tonight or tomorrow.

— Reply to this email directly or view it on GitHub https://github.com/copperdogma/dnd-phb-5e-index/issues/8#issuecomment-156841302 .

-=Cam

ndguarino commented 8 years ago

I got a bit of progress done on the web format, though admittedly not as much as I had hoped. Live search and basic navigation, as well as a full index, are in, but the CSS is extremely basic. Wanted to pass it by you for an initial glance (and see how you envisioned it being styled) before I cleaned things up, however.

My main concern at this point, admittedly, is the stack and methodology I'm using here. While it should be fairly self-expanitory, I also realise some people aren't the biggest fans of React or Webpack.

The code is committed at https://github.com/Falanyx/dnd-phb-5e-index/tree/searchable-html/html and there's a live version up at http://falanyx.com/dnd-phb-5e-index/html/index.html (with a statically pre-rendered version at http://falanyx.com/dnd-phb-5e-index/html/build/index.html). Mostly just making sure you'd be fine with the stack before I finish up and send a pull request, but any input or requests are more than welcome as well.

copperdogma commented 8 years ago

Holy crap nice work!

I'm a C# dev myself, and I've done a ton of js work but I usually build everything from scratch, so your stack is a little alien to me, but that's fine. Where did you learn all of that stuff? Do you use that stack a lot?

I love it. And your assumption about an entity displaying its parents and siblings is right, I think. Helps to give context.

You might need a utility that reprocesses the "PHP Index Improved.txt" file -- it's going to change frequently, especially in the next few weeks as people give their feedback and start using it.

I started on one of my own, but all I have so far is the .txt to json to html processor and basic output. No indexing, no search. Yours is already twice as far ahead!

On Sun, Nov 15, 2015 at 9:46 PM, Falanyx notifications@github.com wrote:

I got a bit of progress done on the web format, though admittedly not as much as I had hoped. Live search and basic navigation, as well as a full index, are in, but the CSS is extremely basic. Wanted to pass it by you for an initial glance (and see how you envisioned it being styled) before I cleaned things up, however.

My main concern at this point, admittedly, is the stack and methodology I'm using here. While it should be fairly self-expanitory, I also realise some people aren't the biggest fans of React or Webpack.

The code is committed at https://github.com/Falanyx/dnd-phb-5e-index/tree/searchable-html/html and there's a live version up at http://falanyx.com/dnd-phb-5e-index/html/index.html (with a statically pre-rendered version at http://falanyx.com/dnd-phb-5e-index/html/build/index.html). Mostly just making sure you'd be fine with the stack before I finish up and send a pull request, but any input or requests are more than welcome as well.

— Reply to this email directly or view it on GitHub https://github.com/copperdogma/dnd-phb-5e-index/issues/8#issuecomment-156914484 .

ndguarino commented 8 years ago

Excellent. I'll clean it up tomorrow when I find a bit of time and finish off the todo list, then submit a pull request. Will also fix the json file to match your recent commits and see if I can get auto-generation of the txt/markdown files done so less maintenance is needed in general if a typo is found. Might also try to separate the line numbers from the text in the json file to improve the search if I'm feeling particularly masochistic.

To answer your question, I picked it up writing a few webapps for work (invoice generator, CMS) . React's nature actually makes it quite useful for data-driven, dynamic client-side applications (to throw a few useless buzzwords around). I don't get as much use out of it as I'd like since I'm primarily a Ruby and C++ dev myself, but if you find yourself curious about it with a bit of free time, I do recommend experimenting with React if only because it's actually somewhat pleasant to work with (and is just a client-side view framework, so it can work on top of pretty much any backend).

copperdogma commented 8 years ago

Faaaantastic.

I think this is my first truly open-source project where someone else contributes! Rah! So exciting;) Thanks for helping out!

Cool. I've heard of React just never used it (we use Knockout, which I'm not a huge fan of). React or Angular were next on our list to look at!

On Sun, Nov 15, 2015 at 11:18 PM, Falanyx notifications@github.com wrote:

Excellent. I'll clean it up tomorrow when I find a bit of time and finish off the todo list, then submit a pull request. Will also fix the json file to match your recent commits and see if I can get auto-generation of the txt/markdown files done so less maintenance is needed in general if a typo is found. Might also try to separate the line numbers from the text in the json file to improve the search if I'm feeling particularly masochistic.

To answer your question, I picked it up writing a few webapps for work (invoice generator, CMS) . React's nature actually makes it quite useful for data-driven, dynamic client-side applications (to throw a few useless buzzwords around). I don't get as much use out of it as I'd like since I'm primarily a Ruby and C++ dev myself, but if you find yourself curious about it with a bit of free time, I do recommend experimenting with React if only because it's actually somewhat pleasant to work with (and is just a client-side view framework, so it can work on top of pretty much any backend).

— Reply to this email directly or view it on GitHub https://github.com/copperdogma/dnd-phb-5e-index/issues/8#issuecomment-156928348 .

daggerhart commented 8 years ago

Hi guys,

Found this thread a little late. I just submitted a PR ( #9 ) that includes json and html live search (possible mobile friendly). Check it out. I'm happy to discuss and join forces for ultimate phb indexing!

+1 for the idea of generating the txt & other files from the JSON file once it is baked. +1 for a proper hosted webpage for this project.

Thanks!

copperdogma commented 8 years ago

Nice! Love to json version -- well done. Of COURSE json is the right answer. It always is;)

Falanyx, what do you think of using daggerhart's json as a source and just kill the text parser portion?

I can write something that will dump it to a variety of formats: plain text, Markdown, maybe the MS Word version...

One of my outstanding todos IS to highlight the canonical page for each reference (instead of having to guess which the main page is: 17, 24, 44, 123-127?). We'd have to update your json format to be able to indicate page(s) as being the canonical reference, but that wouldn't be too bad.

How did you do the json, btw? parser or hand-bombed?

And don't feel bad about missing daggerhart's submission -- I wrote half an online version when his showed up;)

ndguarino commented 8 years ago

Disclaimer: I just read over this after the fact and realised I rambled on a bit. Apologies for the verbosity, but it tends to happen at midnight. TL;DR at the end.

The JSON source by daggerhart is a heavy improvement in many respects, especially the separated page numbers, though I admit complete ignorance to how Isotope does its filtering and as such am wondering if it's recursive (namely because the only reason I had my JSON file flattened was to prevent the need to code a recursive search).

Truthfully, simpler is probably better for this, and daggerhart's solution is far simpler than mine. I might be able to find a mix of the two methods and play around with it, but I will admit right out that the time I thought I had to work on this has in essence disappeared (and this will likely be the case until around the 25th).

The only issue I can think of, personally, with daggerhart's solution other than the searching (which may or may not be covered by Isotope) is rendering the page with all data without requiring Javascript, allowing at least basic use without scripting enabled. However, unless you care about SEO or something crazy like that, which I highly doubt is the case here, the static rendering thing is a non-feature and non-issue as well, as the chance somebody will try to use the index with noscript enabled is slim to none. Also, the JSON being a separate file could increase load times on high-latency high-speed connections (mobile), but an extra 40-200ms means nothing at all (especially when adding framework and library sizes for both daggerhart's and my own implementations, it should equal out to about the same if I minify my files).

This is in contrast with my solution, which has major tooling problems (while I provide instructions for their usage, editing requires a bit more domain knowledge than the alternative, which I personally consider a negative). My version does, however, have the mobile friendliness (mobile-first, though the menu needs a bit of work), jump links, and scrollspy highlighting of the menu, which are also admittedly features easily added to daggerhart's solution.

If the submitted json were flattened, and to be honest I just might flatten it myself, it could be added to my format with minimal changes. The JSON reformatting was already on my todo list, though the new one is far more comprehesive.

In the long run, I'm personally interested in the approach you want to take on this, copperdogma. My solution is probably (read: definitely) overengineered, but does have a few (minor) benefits. Combining the two solutions may be possible, even on just a superficial basis. As for the conversion to text/md/maybe word/maybe pdf, it was something on my todo list as well. Time is always a factor, though it would likely be completely divorced from the web portion anyway, so it's not reliant on any of my existing code when I do find time (if you don't get to it before me, of course.) I'll still be contributing where I can regardless of what you decide, as this has already saved me headaches and frustration with WotC's copyeditors.

TL;DR: The new JSON is awesome, daggerhart's solution is much simpler than mine (and this is probably a good thing), and I can apparently write a short essay in a github issue.

daggerhart commented 8 years ago

@copperdogma To create the JSON it took 6-8 well planned regex find and replaces, and about 2 dozen hand-edits to correct remaining issues. I'm not incredibly happy with the amount of whitespace in the json file, but we can clean that up in future versions.

Concerning canonical pages (and other data), I think we should make a new issue thread where we really nail down the json model for an index section. I had some other ideas while working on it. For example a few items have "See also" information, so should those be separated in the data? I'll post a new issue where we can discuss the model. ( here: #10 )

@Falanyx Isotope isn't really a great solution for this, I just used it because I was familiar with it, and it provided a lot of "fancy" for the bucks. It isn't searching the json data at all, it is actually searching the rendered content using new Regex(), which is cool because it is simple, but possibly not the best way. I'm open to suggestions, but personally lean towards simple solutions.

When I was about 80% done with my javascript I realized this probably should have been done in angular, or anything else really (I've no experience with React yet, but it's on the list).

Main take-away, the HTML file I created is significantly more of am example, or proof of concept, than it is how I think the resulting website "should be". I'm wide-open to suggestions and discussion.

For SEO and speed (static content on first-load), it would be trivial to load the json file with PHP and render the items on the page that way. If you have other preferred languages, I'm happy to learn and help where I can!

Concerning recursion, I like models that allow for it, but don't demand it. At this point we have no reason to do recursion, but if the model for a child item is the same as a parent item, then there is really nothing stopping us from doing it if we need to in the future.

copperdogma commented 8 years ago

I don't have much skill with front-end frameworks. I either use MVC (.NET; not relevant here) or I write my own from scratch.

Do you guys want to hash out what UI framework you want to use? And do you want to work together on it or just decide which of you wants to do it?

I think both solutions are good. For some reason the Isotope version seems slow, though.

I have no desire to optimize for SEO. In fact, sort of the opposite. This project is already probably going to find itself on the wrong end of a C&D from Wizards. I'd rather not make it EASIER for them to find it;) Not that it's hard now.

I don't think we have to worry about speed of loading much, either. It's a reference thing someone will load up when they need it and it will just stay loaded after that. It's not a massive amount of data.

Thoughts?

copperdogma commented 8 years ago

I haven't heard back from either of you in a couple of weeks. We have a couple of options:

  1. You guys can decide to work together to build a mobile version. I just ask that you keep me in the loop.
  2. You guys can decide that one or the other of your solutions is better and work on it together, just one of you work it to completion, or you can give it to me if you want and I'll do the rest.
  3. I can just pick one of your options. If you're cool with this, do you want to be the primary dev or do you want me to do it?
daggerhart commented 8 years ago

@copperdogma Sorry for the lack of response.

Option 3 is the right choice here. It's your project, you should participate in whatever way you want. There are at least two major components to this project; content and code. I suspect you're as likely to get pull requests for updates to the index content/data, as you are to get code pull requests. Ultimately it is up to you to accept and merge those requests or reject them.

To summarize our thoughts so-far:

Our skill-sets:

That's a lot of good skill-set for making a website, we shouldn't have any major problems building whatever kind of site you want.

Also, considering the goal of creating a searchable website is incredibly simple, we have a lot of room for experimentation and error. Don't worry at all about "making the wrong decision" or anything like that. The project is simple enough to pivot easily.

To move us forward I recommend 2 things:

  1. Pick what you want to program, or learn, and build your ideal stack from our skill-set.
  2. Review and consider/discuss my json pull-request. Since I did the content migration programmatically, there may be content errors.

The json doesn't have to be perfect to start, because we will be refining it as we evolve. We should still work towards an ideal/finalized data model in #10).

ndguarino commented 8 years ago

Sorry for the radio silence. Work's been more hectic than I thought it would be. I have to agree with daggerhart about option 3 being the right choice, and pretty much agree fully with almost everything he said. My only suggestions at this point that I would feel reasonable making with how my schedule's would be related to #10, which I will note there.

copperdogma commented 8 years ago

Okie doke let's get this party started.

Let's go with @daggerhart's pull request, fix up the json, and host it.. off github? I think you can do that.

I'm pretty good at HTML/CSS so I may take a crack at refining the look and feel.

@daggerhart, do you have anything else on your to-do list on your solution? If so, I'd love to hear it and help if you want it.