Pagination: ACSS

[dragontheory]

Show/hide based on dividing total number of visible search result rows with total search results returned.

Note: search result rows will have options to view as cards and will be responsive. In that case, we may need to go to an infinite scroll option via CSS media queries.

[bob2517]

Will start off with working with page numbers. Infinite scroll would follow on from getting that working.

[bob2517]

Some thoughts on how to do this.

There are 3 ways to go about this, each with pros and cons.

Currently, using the existing API, or a superhero JSON copy, the only way the pagination spec would work is if we loaded up ALL the records into memory, because there are no paging options on the API at all.

Obviously this isn't very practical.

Ideally we would want to send the API back-end this information to fetch data:

starting record ID ending record ID

But that would only work if the ID was a number.

If the ID was a unique string, we would need to send over this data:

page number starting record ID total number of records that we need for the current page display

That would be ideal. However, most projects would need to set up a back-end API that could handle this. That may or may not be acceptable as a solution, as it isn't a regular thing to do - this is new ground and people don't generally do this sort of JS pagination thing, so they wouldn't have built it into their API.

The other option is complicated to set up from an ACSS point of view, but this would work with regular APIs that have a regular fixed paging facility, which should be most real APIs.

I'm going to lay out that in a sec, once I've got my head around it.

[bob2517]

The variables we would need to keep track of:

page number (integer) number of rows visible (integer) total number of pages (integer) number of records per page that the API can handle (integer) API page numbers required to get all the rows needed on the front-end - ie. we may need records from the bottom of page 2 & the top of page 3 of the API to fetch the records required for our display. So there could be multiple AJAX calls to fetch the data we need. So this variable would be an array. (array) (insert meme)

I think that's it. That would enable this system to work with any API with any paging facility. In theory at least. If it works.

[bob2517]

Next would be finding an API we can use that has a paging facility. Will look around. If not, I'll set one up. Would rather use something that is already available though.

[bob2517]

This method would also work with infinite scroll. There's no real difference in the technique - it's just a case of grabbing the next page contents (which may mean one or more calls to the API - same as the click paging).

[bob2517]

Had a look around and didn't find a decent API with paging that has content that matches what we need, like names and addresses, etc.

I think it's probably going to be easier if I just set up a REST point and split up the superhero JSON files and simulate paging on a server - that's going to be a lot quicker than setting up a database. I can put it somewhere on the internet, like on the ACSS docs site. It won't cost anything then either. We can get a valid CRUD flow then too, simulate errors, etc.

Can you email me the superhero JSON file to support@, or put it in the repo somewhere so I can grab it? Ta.

[bob2517]

If needed, I can set up a database for it later on when it looks like everything's working. Then it will be fully functioning CRUD.

[bob2517]

I'll use the existing app JSON meanwhile, just so I can test the endpoint. And then I'll slot in the superhero records when you send it.

[bob2517]

Great, thanks. I'll grab it from there.

[bob2517]

There's lots of cool data in that file - nice :)

[dragontheory]

Can you email me the superhero JSON file to support@, or put it in the repo somewhere so I can grab it? Ta.

Here is the address I just sent an invite to. https://github.com/dragontheory/dragontheory/tree/main/data

Tutorial I read to do the host the JSON: https://victorscholz.medium.com/hosting-a-json-api-on-github-pages-47b402f72603 They said if you do it right, you can do CRUD operations but I'm not sure I did it right... lol

This is where I generated the random names from Fake data generator https://omatsuri.app/fake-data-generator

Superhero JSON API. I just copied it from here. They also have the JSON broken up into categories and individual JSON files for each supe. https://akabab.github.io/superhero-api/api/

Since this has a lot of detail for each supe (even multiple images for a carousel), I envisioned using this for designing the aside panel. There will be an [ edit ] button and suddenly all the information is editable.

I can do that by putting all the content into a form by default and styling it as if it normal non-editable data. Then when users click [ edit ], it is easy to change the styles so the data looks editable.

Same is true for the normal generated data.

Creating a basic form modeled after the supes data. Unless that is going to get in your way?

[bob2517]

Creating a basic form modeled after the supes data. Unless that is going to get in your way?

No, that's fine, go for it.

[bob2517]

I'm creating a database for the supers which can be queried and edited - it will save time later on.

[bob2517]

I've created the database structure - it's nothing fancy - just enough so we can replicate the API with paging. I'll populate it with data offline, set up the code for the API and push it live once its working. I have an admin area on my offline docs site where I can test it out.

[bob2517]

If you come across anything when making the forms, or have any data related questions, let me know.

[bob2517]

The REST endpoints will be something like:

Get a page of data (GET): https://activecss.org/supers/?perPage=20&page=2

Fetch supe data with a single id (GET): https://activecss.org/supers/?id=23

Save data from a form (with POST variables): https://activecss.org/supers/update

Delete data from the database (with POST variables): https://activecss.org/supers/delete

We may need to host the repo somewhere to get around CORS issues, but we'll see. I have a feeling that we'll need a proper web server to get the ajax stuff working. It's also worth putting it into a secure area, so only certain people can edit the data. I can host it on my server if you like and set up security for it, unless you have somewhere where it can be properly hosted and put behind a secure login.

[bob2517]

If you want to get a proper domain name for it, it could be pointed at my live servers and I can set up a real website behind an nginx admin login. It's not a big deal to do that.

[bob2517]

If we did that, we would need a new PRIVATE repo which would store the back-end code which would provide the API and CRUD back-end code, plus it would keep the actual database (with INSERT commands, which would create the database from scratch) so that it could be replicated on a different server if I suddenly got hit by a bus.

[bob2517]

My live server is super fast too, excuse the pun.

[dragontheory]

I have free accounts with Bluehost, Cloudflare and Netlify. Cloudflare has HTTP3. They all have CDNs. Could we leverage them?

[dragontheory]

I also have the domains https://D7460N.io/ and https://D7460N.dev/

Although https://D7460N.dev/ seems to be in an error state...

[bob2517]

I'll check and see if it's feasible. We do need full control of the back-end though. I'm just wrapping up the import of the supers JSON into a MySQL database and will take a look after that. If those server accounts won't cut it, I'll ask you to point your domain to my live server when I put the API live. Will let you know.

[dragontheory]

I'll check and see if it's feasible. We do need full control of the back-end though. I'm just wrapping up the import of the supers JSON into a MySQL database and will take a look after that. If those server accounts won't cut it, I'll ask you to point your domain to my live server when I put the API live. Will let you know.

I can give you access to those accounts if necessary. No problem at all.

[bob2517]

Cool

[bob2517]

I've put the JSON file into a database now, so the API won't take long to set up now. There are 563 superheroes there in total.

I had a look at the various options and I'm just going to set up the API via the live docs site. It will be fast and reliable and quicker for me to set up. I'll do it so that it mimics the existing supers API, with the addition of "page number" and "records per page" fields. I might put in an "order by" option in the API as well, just in case that is needed.

[dragontheory]

Excellent! Thank you!

[bob2517]

I've written the get supes API call offline and it's tested. I'll do the get supe by ID API call and then put it all live.

[bob2517]

The get supe by ID API call is done. Putting live now - may take a little while as I can't remember how to do the database stuff :). Will post the API commands shortly and then they can be slotted into the project.

[dragontheory]

Wow dude. You are doing great!

Is this hosted on your server, my server, or GitHub?

[bob2517]

It's hosted on the ACSS docs server. It's up there now and seems to work via the URL. Very fast too. I'm going to test it via the project. I'll let you know the URLs shortly.

What do you want to do about the field names? They are all going to be different now. Should I just change them to fit or do you have plans?

[bob2517]

Because it's on my docs server, I can tweak it as much as we want without any restrictions - basically do whatever we want with it. And it will save a heap of time when it comes to getting the CRUD working. The server is basically just waiting for use to send it an HTML form and then I can write the "update" API call really easily.

The paging is all set up with the options that I mentioned, including being able to sort by a bunch of fields, ascending or descending. I'll post the commands in a minute.

It's really not that difficult to do in native PHP. It's having to learn all the third-party tools that do it all for you that actually slow things up, because you have to try and work with the configuration options, which takes the same amount of time to set up as writing it from scratch, if you know what you're doing. Which is why I think it's quicker just to write these things yourself. I find it a lot easier and quicker, ironically.

[bob2517]

The possible fields are as follows:

[
  {
    "id": 1,
    "name": "A-Bomb",
    "slug": "1-a-bomb",
    "powerstats": {
      "intelligence": 38,
      "strength": 100,
      "speed": 17,
      "durability": 80,
      "power": 24,
      "combat": 64
    },
    "appearance": {
      "gender": "Male",
      "race": "Human",
      "height": [
        "6'8",
        "203 cm"
      ],
      "weight": [
        "980 lb",
        "441 kg"
      ],
      "eyeColor": "Yellow",
      "hairColor": "No Hair"
    },
    "biography": {
      "fullName": "Richard Milhouse Jones",
      "alterEgos": "No alter egos found.",
      "placeOfBirth": "Scarsdale, Arizona",
      "firstAppearance": "Hulk Vol 2 #2 (April, 2008) (as A-Bomb)",
      "publisher": "Marvel Comics",
      "alignment": "good"
    },
    "work": {
      "occupation": "Musician, adventurer, author; formerly talk show host",
      "base": "-"
    },
    "connections": {
      "groupAffiliation": "Hulk Family; Excelsior (sponsor), Avengers (honorary member); formerly partner of the Hulk, Captain America and Captain Marvel; Teen Brigade; ally of Rom",
      "relatives": "Marlo Chandler-Jones (wife); Polly (aunt); Mrs. Chandler (mother-in-law); Keith Chandler, Ray Chandler, three unidentified others (brothers-in-law); unidentified father (deceased); Jackie Shorr (alleged mother; unconfirmed)"
    },
    "images": {
      "xs": "https://cdn.jsdelivr.net/gh/akabab/superhero-api@0.3.0/api/images/xs/1-a-bomb.jpg",
      "sm": "https://cdn.jsdelivr.net/gh/akabab/superhero-api@0.3.0/api/images/sm/1-a-bomb.jpg",
      "md": "https://cdn.jsdelivr.net/gh/akabab/superhero-api@0.3.0/api/images/md/1-a-bomb.jpg",
      "lg": "https://cdn.jsdelivr.net/gh/akabab/superhero-api@0.3.0/api/images/lg/1-a-bomb.jpg"
    }
  },

Plus per page, page number and total pages, which I actually forgot to add - I'll do that now.

[bob2517]

So in the code it will be

$item.name $item.powerstats.intelligence

etc...

[dragontheory]

Oh wow. OK. Umm... I don't know. Like you said, maybe go with whatever is industry standard?

[bob2517]

No, I mean, what fields do you want to show in the project?

We've currently got regular people in there, not superheroes. Which fields do you want to show in the rows list and which fields do you want to show in the aside?

[dragontheory]

...since that is what this will be a demo for. Our audience, at least for the end of the month are other DEVs who will be looking to either start new with this or integrate into existing systems.

[bob2517]

Not sure I'm getting what you're saying. Could you clarify?

I was just hinting that I don't know about how best to set things up on the data back-end side.

I am always trying to think of who my audience is. So when I thought you were asking me about what form elements to show, I was just trying to think of who will be seeing this app at the end of the month for my new position on the new team. So I was just thinking that as you said, standards are so important so that other DEVs can pick up and run with what we do.

Sort of just rambling. You know all this. Never mind... lol

[dragontheory]

No, I mean, what fields do you want to show in the project?

Oh. Gotcha.

Aside panel... All of them I guess. I may not show them all at once but they should be for when I do need them for the UI. Have accordion sections that group the stats maybe. I need to show that this app can handle LOTS of data well. Not only in speed but visually. Thus useful to management.

For search result rows... ID, alias, real name, current status, current org/affiliation, current location, power rating/rank?

[bob2517]

Sounds good to me. Ok - will try and get this done in the next half hour and onto the branch. It will be rough, but it will be a good end for the day.

[dragontheory]

Sounds good to me. Ok - will try and get this done in the next half hour and onto the branch. It will be rough, but it will be a good end for the day.

Dange dude. You've done enough for a week!

[bob2517]

The rough part will be the pagination. The rest I've done a bunch of times and shouldn't take that long. I'd rather be on top of it than not, as there is always something that slows things up unexpectedly somewhere. I spent 5 hours learning how to draw and animate a triangle in an SVG on Friday. So my speed can get quite random - it depends whether I know how to do it or not.

[bob2517]

I've been building UI interfaces for databases since 1989, so I should vaguely know what I'm doing by now :) Before the internet that was literally my job. All I did was build pretty boring UIs and database back-ends, then when the internet came out I learned how to do it on there.

[dragontheory]

I spent 5 hours learning how to draw and animate a triangle in an SVG on Friday. So my speed can get quite random.

Oh my goodness, that is so true!

In a way, that is exactly what I am trying to mitigate against for next month with this new position. Trying to ask myself, what are DEVs going to want to know and need and change to make it work as a new project or integrate into an existing project. That is why I want to EVENTUALLY (not this time around), make demos with Angular, Vue, and React. The more I have ready for them to take and run with, the more chance there is for them to adopt.

[bob2517]

Right, yeah.

[dragontheory]

I've been building UI interfaces for databases since 1989

Was the Y2K bug a big deal over there?

I remember Jan 1 2000 was my first day at my first UI programming job. Was a usability tester before that. It was chaos day one. Turned out to be nothing big at all. Not sure if that was thanks to all the C and Cobalt old timers or what. But it turned out to be such a dud. News wise.

[bob2517]

Here are a couple of URLs. If you look at them in Firefox - look at the response column for the get call, you will see the JSON object.

Gets a supe by ID (the id is self-explanatory): https://activecss.org/get-supe.php?id=200

Gets a list of supes: https://activecss.org/get-supes.php?perPage=5&page=1

The options are: perPage (eg. perPage=10) - defaults to 10 if not specified. page (which is page number) (eg. page=3) - defaults to page 1 id not specified. desc (which equals 1 to sort the database in reverse) (eg. desc=1) - defaults to sort the database in the regular order. orderBy (which can one of the following: 'name', 'intelligence', 'strength', 'speed', 'durability', 'power', 'combat', 'gender', 'race', 'height_inches', 'height_centimetres', 'weight_pounds', 'weight_kilos', 'eye_color', 'hair_color', 'full_name', 'publisher', 'alignment',)

[bob2517]

I've been building UI interfaces for databases since 1989

Was the Y2K bug a big deal over there?

I remember Jan 1 2000 was my first day at my first UI programming job. Was a usability tester before that. It was chaos day one. Turned out to be nothing big at all. Not sure if that was thanks to all the C and Cobalt old timers or what. But it turned out to be such a dud. News wise.

Yeah, I think I changed one piece of code out of all the things we had to look at. I was working for a call centre at the time, building database systems in a team of a dozen programmers. We all spent time reviewing the existing systems, but it wasn't much of a big deal for us either. Nothing happened really :) I guess it was better to be safe than sorry.

[bob2517]

As suspected, I'm running into CORS issues with the new API and using the project from the git directory. It's probably the way my server is set up or something, as it was working fine with the public API. I need to investigate it.

[bob2517]

I got the CORS thing working - I had to set some headers for the JSON response, which I could do in PHP. So that's fine and live. Just need to update the fields now, and then will call it a day.

[dragontheory]

Wow. OK dude. That was a full day. I hope to get the form out in one day... lol