bmlt-enabled / tomato

Aggregates the data from all known root servers while acting like a root server itself.
https://aggregator.bmltenabled.org/main_server/semantic
3 stars 5 forks source link

service discovery not working #12

Closed dgershman closed 6 years ago

dgershman commented 6 years ago

several plugins/apps use BMLT service discovery, they are currently not supported with tomato. they are:

dgershman commented 6 years ago

I just ran the sandwich test suite against tomato. This is a good litmus test for getting things working. https://github.com/radius314/sandwich/blob/master/test/test-spec.js#L4

jbraswell commented 6 years ago

The implemented endpoints are now working with the semantic workshop. Sandwich's test-spec is really helpful for a comprehensive list of endpoints. My next step is to grep the source code of the mentioned projects for unimplemented endpoints, create issues for them, and then implement them.

dgershman commented 6 years ago

semantic workshop working good, csv export working good bread working good

dgershman commented 6 years ago

@littlegreenviper is going to send you @jbraswell a TestFlight invite to two mobile apps that are going to point to the test endpoint of tomato.

LittleGreenViper commented 6 years ago

Gimme a day or two. TestFlight https://developer.apple.com/testflight/ wants to have the beta app in place before sending out invites. I'll set this up tomorrow morning.

I will need a valid email. I don't have an email for Jonathan.

Chris Marshall

Principal chris@littlegreenviper.com mailto:chris@littlegreenviper.com https://littlegreenviper.com https://littlegreenviper.com/ Little Green Viper Software Development LLC

On Mar 12, 2018, at 9:10 PM, Danny Gershman notifications@github.com wrote:

@LittleGreenViper https://github.com/littlegreenviper is going to send you @jbraswell https://github.com/jbraswell a TestFlight invite to two mobile apps that are going to point to the test endpoint of tomato.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jbraswell/tomato/issues/12#issuecomment-372512714, or mute the thread https://github.com/notifications/unsubscribe-auth/Ab9HLnurzd4-Ev6CRt-zeSKlXIHPxmaVks5tdxx3gaJpZM4Sl_aa.

dgershman commented 6 years ago

👍 , we are swapping out the hostname , will send it over shortly

jbraswell commented 6 years ago

Awesome. Just sent you my email address on fb messenger.

dgershman commented 6 years ago

The satellite plugin is choking on this request /main_server/client_interface/xml/GetLangs.php

jbraswell commented 6 years ago

Great. That’s an easy one to implement. Will get it done this week, maybe tomorrow.

dgershman commented 6 years ago

tabbed plugin works

LittleGreenViper commented 6 years ago

Yeah. This looks like a mess. It was caused by the new formats response. It may not be so easy to fix.

LittleGreenViper commented 6 years ago

OK. This is fixed. My bad.

LittleGreenViper commented 6 years ago

It was actually caused by the new midnight code. I have no idea why it didn't manifest in my testing. It should have. PHP is annoying, as you don't find bugs that would normally be compile-time bugs in other languages.

jbraswell commented 6 years ago

Both the json and xml GetLangs.php endpoints are now implemented, but only the json implementation has been deployed to production. I’ll deploy the latest version this evening when I get home.

Right now it is hard coded for English. I’m still figuring out what exactly is required to support different languages from the API standpoint. I think it’s just a matter of importing formats for multiple languages from root servers using GetFormats and the lang_enum parameter, and then serving them up, but it might be more complex than that.

LittleGreenViper commented 6 years ago

The standard satellite doesn't like Tomato. You'll need to provide a copy of the XML /client_interface/serverInfo.xml file. May need other stuff, as well, but that's the first thing.

LittleGreenViper commented 6 years ago

https://tkddevel.com/bmltwork/index.php/tomato-quicksearch/

jbraswell commented 6 years ago

serverInfo.xml is already implemented. Is the information I’m serving from serverInfo.xml insufficient?

LittleGreenViper commented 6 years ago

I'm not sure. I'll need to dig in and debug the process. I know the standard satellite needs some XML to get started.

LittleGreenViper commented 6 years ago

Here are the XML calls made at startup (My localhost server, but substitute your own): http://bmlt-server.localhost/client_interface/serverInfo.xml http://bmlt-server.localhost/client_interface/xml/index.php?switcher=GetFormats http://bmlt-server.localhost/client_interface/xml/GetLangs.php http://bmlt-server.localhost/client_interface/xml/GetServiceBodies.php They all need to return valid XML.

jbraswell commented 6 years ago

Okay. That last one still needs to be implemented. I’ll get on that.

jbraswell commented 6 years ago

I implemented GetServiceBodies.php, fixed a bug with how get_formats_only worked, and made a change to allow double slashes (main_server//) in the URL. https://tkddevel.com/bmltwork/index.php/tomato-quicksearch/ seems to load up without issue now. It's a bit slow to load because it's pulling down 21000 meetings, but it seems to load without any javascript errors.

jbraswell commented 6 years ago

I see that https://tkddevel.com/bmltwork/index.php/tomato-quicksearch/ has a "search by text" feature. I'm assuming it's using SearchString and related options. I have SearchString working when you're searching for an address, but I have not yet implemented full text searching. This will be non-trivial, but I did research all of postgres's full-text search features last weekend, and it's definitely doable.

LittleGreenViper commented 6 years ago

Awesome!

Great stress test!

BMLT_QUICKSEARCH and BMLT_TABLE are supposed to be more tightly focused anyway, so this is good.

LittleGreenViper commented 6 years ago

The string search in the BMLT is fairly primitive. I'll bet you can beat it.

jbraswell commented 6 years ago

I have issue https://github.com/jbraswell/tomato/issues/9 open for implementing the search stuff.

jbraswell commented 6 years ago

@LittleGreenViper Will you verify that my summary of the functionality at https://github.com/jbraswell/tomato/issues/9 is accurate? I'm going to start looking at this.

jbraswell commented 6 years ago

Okay, I implemented the SearchString + SearchStringAll use case. It seems to work pretty well with that UI.

jbraswell commented 6 years ago

All SearchString options are now implemented.

dgershman commented 6 years ago

Looks like the satellite is choking on /main_server/client_interface/xsd/GetSearchResults.php

jbraswell commented 6 years ago

I'll throw something together for that this week.

I'll probably want to spend some time brushing up on xsd, and then looking into all of the fields described in the xsd. I'd rather get this one right than just throw something in there.

jbraswell commented 6 years ago

Oops... accidentally closed and reopened the issue.

dgershman commented 6 years ago

Ok this is the last one, after this we can retire sandwich.

jbraswell commented 6 years ago

Looks like I'm not including the locationInfo xml element at the end of a xml-based GetSearchResults query.

<locationInfo>
   <search_average>
      <location>
         <latitude>33.318144509189</latitude>
         <longitude>-83.986601188048</longitude>
      </location>
      <radius>
         <miles>1300.0423826749</miles>
         <kilometers>2092.210208134</kilometers>
      </radius>
   </search_average>
   <search_center>
      <location>
         <latitude>32.84162245</latitude>
         <longitude>-99.826429</longitude>
      </location>
      <radius>
         <miles>NAN</miles>
         <kilometers>NAN</kilometers>
      </radius>
   </search_center>
</locationInfo>

I am also not respecting get_used_formats or get_formats_only for xml-based GetSearchResults queries. For some reason I thought those were json only. I can add those easily.

Once I get both of those in place, I'll throw the xsd up there.

dgershman commented 6 years ago

It just return a bunch of XSD https://na-bmlt.org/_/sandwich/client_interface/xsd/GetSearchResults.php

dgershman commented 6 years ago

You are returning a 404.

jbraswell commented 6 years ago

Right. I'm saying I don't want to throw the XSD up until I'm actually returning the correct XML. By inspecting the XSD, I realize I had some bugs in my XML implementation.

dgershman commented 6 years ago

Ok, sandwich just returns the first result from a single root https://github.com/radius314/sandwich/blob/3e7e6167e596d213abb3b1cdc47e7e020a5b3061/server.js#L209

LittleGreenViper commented 6 years ago

Don't retire Sandwich quite yet. Remember that's there still two apps out there that refer to it. If Tomato is a valid "drop in" replacement, then we can point the old Sandwich Root URI to Tomato.

Once the new versions of the app are released, then we're shiny.

Chris Marshall

Principal chris@littlegreenviper.com mailto:chris@littlegreenviper.com https://littlegreenviper.com https://littlegreenviper.com/ Little Green Viper Software Development LLC

On Mar 13, 2018, at 10:35 PM, Danny Gershman notifications@github.com wrote:

Ok this is the last one, after this we can retire sandwich.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jbraswell/tomato/issues/12#issuecomment-372885086, or mute the thread https://github.com/notifications/unsubscribe-auth/Ab9HLjNOk4yL7F_b733s5DJ9CVPytcL8ks5teIHngaJpZM4Sl_aa.

jbraswell commented 6 years ago

Last night, Danny had the idea of just using DNS to retire sandwich. I set up tomato to accept the same URL paths at sandwich, and it seems to work. When it’s time, we can just CNAME sandwich to tomato.

LittleGreenViper commented 6 years ago

Sounds good.

jbraswell commented 6 years ago

Okay, I added both the GetSearchResults and GetFormats xsds

dgershman commented 6 years ago

service discovery is working

dgershman commented 6 years ago

satellite plugin is working, i consider this closed.