search

opencaching / opencaching-pl

The source code of Opencaching.PL (and some other domains)
https://opencaching.pl/
GNU General Public License v3.0
22 stars 33 forks source link

Old issue OCNA migration to new code #1973

Closed TermiteHunter closed 3 years ago

TermiteHunter commented 5 years ago

Hello and forgive me if speaking out of turn.

It is now Feb 2019 (2 years since the last comment here on the migration) and we are still operating on an even more broken site. There are failures on our site that affect all users and discourages new members.

The attempt to bring OCNA into the new PL code failed long ago when we were abandoned and forgotten. Attempts to get help with a PL or DE styled update by Mr-Yuck have been met with a few promising ideas from various individuals only to be abandoned before they begin.

The OC Community is about to lose OpenCaching North America. (Mexico, US, Canada and parts of Central America) The site looks and operates poorly, an embarasment to the OC community as a whole. This can not continue. I find that I can no longer encourage others to explore OCNA in it's cuurent state and have even declined to list my own caches on OCNA (and I am an Admin there)

Please contact Mr-Yuck and re-instate the OCNA migration to an updated version of the OC code so that we may continue as a valued and valid OC site. Need is URGENT. Admin at OCNA are preparing to allow this node to join Navicache and become a non-entity if help is not recieved soon.

Action is needed. The site is failing. (maps fail. search fails, registration fails) Admin is discouraged. (too many promises with no follow-up and abandonment) Use is plummeting. (new users down, listings down, logs down) Failure is imminent. (by code failure, lack of use, or intentional failure to renew domains or hosting for a failing site)

If the goal is to have nodes in all parts of the world, The community is failing a huge part of that world. Save OCNA NOW

kojoty commented 5 years ago

hi @TermiteHunter

Let's open a new discussion and try to conclude where we are now.

First: OCPL as a whole has also its problems. There are only a few developers who works on maintenece and development of OCPL code. We are in a long run to the refactoring of the "old" code and there are still much work to do.

Second: What is your (OCNA) current situation - how many technical people are there now - what kind of support you need and expect and what you can ensure yourself. I mean if there is someone who has time and skills to cooperate with us in this process of migration OCNA to OCPL code.

@Mr-Yuck are you there? What's your opinion?

@deg-pl you have experience with resurrection of OCUK. Do you think we are able to do the same for OCNA?

@deg-pl @harrieklomp @following5 @andrixnet @mzylowski What do you think? Do you think we can cooperate now to help with OCNA migration to OCPL code?

If you decide to help I'm in.

deg-pl commented 5 years ago

I'm pro. I can help. But we have one big issue - we don't support states and US post codes. Nor in caches and search engine.

Rest should not to be a big problem. I can make new OC NA server and probably to migrate existing data. I'm only busy for a few next days.

following5 commented 5 years ago

Subissues:

Special tables in OCUS database:

harrieklomp commented 5 years ago

@deg-pl @harrieklomp @following5 @andrixnet @mzylowski What do you think? Do you think we can cooperate now to help with OCNA migration to OCPL code?

Yes, let us help OCNA with this great OCPL code? This shows that the staff behind this great code also are great :-)

TermiteHunter commented 5 years ago

Thanks for the response. I am trying to get MrYuck involved. years of no results has left him a non-frequent visitor to OCNA and Geocache related e-mail accounts. He will pop up sometime.

To First question: We are aware of ongoing issues with PL. That was stated as a partial reason the upgrade was terminated 2 years ago. Let's face it, There will always be some issues to contend with but a basicly functional site with at least the appearance of being maintained is prefferable to OCNA as it is.

To second question: We have NO tech people. We can find the time but have no expertise which is partially why we are this situation. We just want a site that members can actually use that looks to be maintained. Graphics for some items not used in Europe were already provided.

I'm pro. I can help. But we have one big issue - we don't support states and US post codes. Nor in caches and search engine. We currently have this ability in the old code. I believe that much of this may have already been done or at least started previously. It doesn't have to be by zip code (it would be nice and preffered) Something similar would be better than only a single page being returned with following pages failing to resolve. This occurs on all searchs, zip or not.

Thanks again.

kojoty commented 5 years ago

@TermiteHunter

we need to know if we can start:

then:

TermiteHunter commented 5 years ago

I just checked to see if I still had access to our server. Apparently I do not.MrYuck has the domain and server accounts.Andrei? Was helping before. He may still have the info for access. I will redouble efforts to get MrYuck to appear.On Feb 16, 2019 5:29 PM, kojoty notifications@github.com wrote:@TermiteHunter we need to know if we can start: do you have access to any hosting account?who controls the OCNA domain?do you have an access to current server - we will need to copy pics etc. then: @deg-pl is able to prepare new server configuration for youin the meantime we try to identify and if possible fix the incompatibility in code and configuration

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

Mr-Yuck commented 5 years ago

C'mon TH, it's not THAT difficult to get me to appear. Andrei was helping us, and more specifically, we were talking about a quick "OCUK type" fix. But I haven't heard from him since mid October. I believe he does have a test server running with OCUS mostly migrated. We have a web hosting company out of Montreal Canada, and of course I have access to the server, which I interface with with cPanel. I haven't used ftp in probably 3 years, although you can set it up with cPanel. As noted at the very bottom of the page on opencaching.us, it is currently running with Litespeed web server (not apache), CentOS (not Ubuntu), Mariadb (not MySQL) and Exim internet mailer. Let me know what you would need. And I think Andrei should definitely be consulted first, to see if anything he's done may be of use.

kojoty commented 5 years ago

@andrixnet are you there? (Andrei is here but it seems he has been busy for last few months)

I think that the best solution would be the same as for OCUK: prepare the new server - I don't know how many traffic you have right now but I expect that cheap VPS would not cost much more. In my opinion this is much better because staying with completely different software stack can caused additional issues, which can be difficult to debug for us - I believe @deg-pl agrees with this - am I right?

In the meantime today I will try to review issues which andrixnet marked as a important for OCNA today.

deg-pl commented 5 years ago

Yes. I agree with @kojoty Best option is new VPS in KVM virtualisation (not OpenVM!!!). In google I've found many cheap VPS in USA (under $10/month). OC UK use cheap SmallVPS https://www.arubacloud.com/vps/virtual-private-server-range.aspx

Let's try to find which parts of our code doesn't fit OC NA and set priorities. I think for example, that "states" support is important (maybe our regions will fit?) but apporiving all caches by OC Team is not such important for now.

Mr-Yuck commented 5 years ago

Andrei (who we definitely must consult with) had actually mentioned last October that he had enough server space to host us for free? We were originally set up on the U.S. web hosting Company Rackspace, which was costing $40 U.S. a month! We pay $16 U.S. a month now, and this company (Kickassd) given server access to Rackspace, moved the site over for free. I concur with all migration issues mentioned here, and I will say every one of them have been discussed with Andrei at one time or another.

andrixnet commented 5 years ago

In part it is my fault. I've been in contact with one of OCNA admins (Mr. Yuck I think, I also knew him as "Jim"). He has given me access to OCNA hosting several years ago and I helped him out with some small bugs/problems over time. OCNA was based on a modified version of the old OCUK fork. I suggested migrating to current opencaching-pl code, and to that end I initiated project OCNA migration here. Also, highly related, is project Well defined data types because of the differences in geocaching activity and many remnants of old DB based data types in opencaching-pl code.

While trying to make heads and tails of the migration necessities, my progress was slow. In 2018 I set up a devel server of my own for both OCRO and OCNA, on which to test my stuff. Since then I lost contact with Mr. Yuck. Also, since the birth of our second child, my time for this project shrunk to zero (OCRO also severely affected; @following5 I still have to implement some of your instructions).

andrixnet commented 5 years ago

That's it with the history. Now to the practical.

As I said, I have a devel server up and running for OCRO and ready for OCUS, all but the database conversion. I can offer it as hosting for OCUS working on opencaching-pl code at no cost.

The existing hosting of OCUS is big mess, at least configuration wise. I still have access to it. I have a complete backup of all OCUS resources (files and databases):

I think I can make another set, as long as the Kickassd hosting is still up and I have access.

andrixnet commented 5 years ago

Technical-wise, project OCNA migration describes most requirements. I'll try to summarize them again here, to the best of my knowledge:

  1. OCNA specific cache types

  2. OCNA specific attributes This was hardest for me, since cache types, sizes, attributes and a few other resources are still scattered in both code and database for opencaching-pl

  3. region assignment based on US states (and Canada and Mexico). I have the OCUS database, which contains geographical data similar to our nuts_layer and nuts_data resources. This should not be too difficult to convert. I made partial progress, keep in mind that I also wanted to convert al geographical data to polygons (instead of linestrings) so that ST_WITHIN MySQL/MariaDB function returns the proper result. (see here https://github.com/opencaching/opencaching-pl/issues/120)

  4. detect/store/search by ZIP code. OCUS database contains similar geographical data regarding ZIP codes, this would be a totally new implementation in opencaching-pl. Either use that data, or add a dependency on an online service, other then Google, of course, due to costs. This last one, while I understood may be important to US users, used to using ZIP codes, is hardest and most time consuming to implement.

andrixnet commented 5 years ago

Pssst, the kids are sleeping, so I'm busy writing here. Harrie kept me informed from time to time about OC developments.

I'm looking up the stuff I already did for OCUS on my computer and reviewing it.

following5 commented 5 years ago

@andrixnet

  1. OCNA specific attributes This was hardest for me, since cache types, sizes, attributes and a few other resources are still scattered in both code and database for opencaching-pl

Cache sizes meanwhile are completely migrated to code; you just need to add Config/geocache.us.php to configure the OCUS sizes.

For attributes I can do the same, as soon as the DB update system is working at OC RO (see https://github.com/opencaching/opencaching-pl/issues/1943#issuecomment-461451833).

Then only the cache types issue is left => #860.

Mr-Yuck commented 5 years ago

@andrixnet Mr.Yuck is definitely Jim. :-)

TermiteHunter commented 5 years ago

We no longer have the bitcaching site.Lost the domain some time ago.Bitcaching is now a crypto currency exchange.We do still use bit caches as a cache type. On Feb 17, 2019 8:03 AM, andrixnet notifications@github.com wrote:That's it with the history. Now to the practical. As I said, I have a devel server up and running for OCRO and ready for OCUS, all but the database conversion. I can offer it as hosting for OCUS working on opencaching-pl code at no cost. The existing hosting of OCUS is big mess, at least configuration wise. I still have access to it. I have a complete backup of all OCUS resources (files and databases): OCUS opencaching siteforumwikiblogbitcaching page These were made december 2018. Based on these I should be able to recreate them on a separate hosting, should Kickassd hosting cease to function. I think I can make another set, as long as the Kickassd hosting is still up and I have access.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

andrixnet commented 5 years ago

cache_status is another table containing data type definitions waypoint_type is also. log_types is also

following5 commented 5 years ago

cache_status is another table containing data type definitions

Also log_types and waypoint_type. But I don't think that any of these have special contents at OCUS, which need special care when migrating to current OCPL code. (log_types definitely has not ... whoop some minor differences found, but not relevant for migration). Attributes also are uncritical - GPX already is prepared for OCUS attributes, and the rest of code uses the cache_attrib table contents.

I think the cache types are the only of those entities that need special migration.

andrixnet commented 5 years ago

OKAPI attributes.xml might need some tending (?)

following5 commented 5 years ago

Current OKAPI installation at OC US ignores all the attributes - it returns an empty attribute list for all caches. This will not change when migrating to current OCPL & OKAPI code. US attributes then may be added to OKAPI after the code migration.

andrixnet commented 5 years ago

OKAPI attributes.xml might need some tending (?)

Mr-Yuck commented 5 years ago

Does anyone need anything from me? Do you still have server access, @andrixnet ?

deg-pl commented 5 years ago

@Mr-Yuck could you export DB (i.e. via mysqldump) and send to me using any network storage like Google Drive or Microsoft OneDrive? And send link to me (deg at best dot pl)

andrixnet commented 5 years ago

Yes, I do have access. @deg-pl I'll make a new export today.

andrixnet commented 5 years ago

@deg-pl will my hosting space help? (and partial instance "work-in-progress") ? I can give you access to it.

andrixnet commented 5 years ago

@deg-pl export done, some files still loading. I've emailed you access to my hosting space where you can find all the files. For any questions please contact me via email or FB mess.

andrixnet commented 5 years ago

A partially running OCNA copy using opencaching-pl code can be found here: http://ocus.dev.andrix.eu

I managed to finally import most of the database and have it partially working. It is under development, yet accessible at this time without further restrictions. If a supplementary password protection is desired, just let me know.

Things that do not work:

Technical; @deg-pl :

Old OCNA database contains some tables which I believe are no longer needed, however:

These geo tables might be convertable to the nuts_* data opencaching-pl code uses.

Mr-Yuck commented 5 years ago

Good progress. Does anyone need anything from me?

andrixnet commented 5 years ago

@Mr-Yuck technically no, at this time. But we'll keep you updated.

andrixnet commented 5 years ago

Progress on OCUS mirror devel installation.

I would like to ask some US cache owners to login to this server and check their caches.

IMPORTANT! This is a development, very intermediate work, by no means production ready. But it is partially workable and for testing.

More to come.

All feedback welcome.

TermiteHunter commented 5 years ago

I tried it. Looks to be in good basic order so far. Was able to view listed caches and navigate to various cache pages. View recent logs etc. No RO caches, all US. RULES sis go to the OCPL Wiki rather than US Wiki. I was unable to log in. Tried password reset but did not get the reset e-mail. I'll watch for the e-mail to reset password.

harrieklomp commented 5 years ago

I was unable to log in. Tried password reset but did not get the reset e-mail. I'll watch for the e-mail to reset password.

Did have the same problem. It did send the mail to other e-mailaddress. I remember that i changed it a while ago. Can login now :-)

TermiteHunter commented 5 years ago

Still no reset email for me. Tried reset twice (all mail checked)

andrixnet commented 5 years ago

@TermiteHunter please contact me directly. @yahoo, @gmail (email or hangouts), skype, xmpp.jp

TermiteHunter commented 5 years ago

So where are we on this "upgrade"? I have not been able to get the password reset (since the last indication that I should) I have asked another OCNA Admin to take a look but he will be unable until sometime this week.

kojoty commented 5 years ago

From the other side: guys, I've reviewed the current OCPL cache-location code and we are more-less ready to support US data. Do you have any gis data for America?

I mean:

Then I will integrate it with our current data - I would like to import also the newest european data from NUTS-2016 - it seems that all of these should works together.

andrixnet commented 5 years ago

@kojoty Yes, there is. SQL tables with LINESTRING shapes. A bit simpler then NUTS data, less hierarchy, in separate tables for US, MX, CA. It would be of great help if you could sort them out, also sort out some other migration data. At some point I am not sure how to proceed.

I can show you my work. I'll contact you privately with details.

@TermiteHunter there seem to be more email subsystem problems that I will have to sort out. Do you have fellow cachers with a yahoo account to ask to try it out? (it worked for me, so far, with yahoo).

Mr-Yuck commented 5 years ago

I will try with a yahoo account.

andrixnet commented 5 years ago

In contact also with "Bon Echo" OCNA admin. Fiddled a bit more with email subsystem settings to improve deliverability.

harrieklomp commented 5 years ago

Just updated the software with "git pull", so we can see the actual state on http://ocus.dev.andrix.eu/index.php .

harrieklomp commented 5 years ago

A small update for OCUS devel site.

These where the easy things to do. The harder things i leave to the more experienced persons :-)

kojoty commented 5 years ago

@harrieklomp and the rest of collegues :)

These where the easy things to do. The harder things i leave to the more experienced persons :-)

Please create one updated list of issues which we still have in this subject

andrixnet commented 5 years ago

  1. Most important of all: cache types. OCUS has a larger list of cache types then most other nodes and an additional problem is that some cache type IDs overlap. This I researched thoroughly before and the complete documentation can be found here: https://wiki.opencaching.eu/index.php?title=Cache_types There are several implications:

    • there are overlapping cache type ID numbers between OCPL, OCDE and OCUS.
    • IMO all cache type IDs must be unique, therefore I propose changing IDs for types that have overlapping IDs and assign new unique IDs to new types from OCUS
    • code should be updated to reflect these changes.
    • when code is updated, a database update must be deployed as well, to change the resolved duplicate IDs on all opencaching-pl based nodes.

  2. Support for mascot image in header, besides the logo. oc1

  3. Additional top menu items and their respective pages (from http://www.opencaching.us/index.php) I think Harrie can handle it. oc2

  4. Home page welcome text I think Harrie cand handle it.

  5. Additional items in MAIN MENU and their respective pages oc3

  6. Additional links on the left menu:

    • OC network links
    • donate link
    • mascot

  7. very important game parameter:

    • OCUS counts event "attended" to total finds. IMO this should become a per/node configurable parameter, by default OFF, but set to ON for OCUS.

  8. USA, Canada and Mexico geo data import from old database in a compatible way to our NUTS data. Part of it I am trying to handle myself, work in progress. Where I get stuck, I'll ask for help.

  9. Search by ZIPcode. ZIPcode geo data import from old database is easy, but support for search by it (PHP code) I cannot handle.

  10. USA, Canada, Mexico regions definitions. Probably part of 8) Also correlation with existing cache data

  11. setup of Powertrail module. I think Harrie can handle it.

  12. set website icon (see https://github.com/opencaching/opencaching-pl/issues/1547 )

  13. setup of cron jobs I will handle it.

  14. setup of automatic updates To be done after migration and move to production.

  15. french and spanish translations

  16. how to handle differences between british english and american english. If these differences account only for a few cache attributes, then perhaps the attribute can be named something like this: Quick cache / One-minute cache

If there are more such differences, not only attribute name (which can be handled easily), then another solution must be designed. The only one I can think of is to maintain 2 sets english strings (en-us and en-gb).

  1. challenge caches is a OCUS specific thing that they did upon on unknown cache type. Perhaps this deserves it's own type.

  2. the page "phone apps for OC" (from main menu) I am sure might be of interest to all OC nodes

  3. Perhaps a simple online editor for articles, similar to news, available for node admins (of use for all OC nodes, see https://github.com/opencaching/opencaching-pl/issues/1946 )

These are most of what I can think of at this time.

andrixnet commented 5 years ago

Oh, and another very important issue:

  1. convert passwords from old cca 2009 opencaching-pl code to -current authentication scheme and the new salt and hash scheme.

This is very important to allow seamless transition for the users, instead of requiring each user to do a manual password reset.

harrieklomp commented 5 years ago 
1. convert passwords from old cca 2009 opencaching-pl code to -current authentication scheme and the new salt and hash scheme.

This is very important to allow seamless transition for the users, instead of requiring each user to do a manual password reset.

This converting does go automatic. If logged-in or generate new password there will automatic will add a "salt" and "hash".

TermiteHunter commented 5 years ago

Thanks to Harrieklomp I have managed to login to the Dev site. My email was changed and somehow that allowed me to login using my regular credentials without a password change.

I have yet to explore the site fully. This may have been mentioned before but I did not see it, The main page MAP OF CACHES still goes to the non-functional google map.

Thanks to all those participating in the OCNA update. You are doing great work.

harrieklomp commented 5 years ago

The main page MAP OF CACHES still goes to the non-functional google map.

If i remember correct there is only one page with old maps. See http://ocus.dev.andrix.eu/myroutes_add_map2.php. All other maps are the newest version.

Perhaps used by accident a link to another site? BTW, in the Menu it is called "Cache map"

TermiteHunter commented 5 years ago

Must have been some sort of other issue that gave me the old Google map. I now wee the correct current map when clicking Cache Map.

Low priority: QR code needs a OCNA version. currently shows OC.PL as url and has the OC sun logo. Request that the OCNA North America version logo and Opencaching.us url be used.

Future suggestion for all OC very low priority.: Foreign waypoints boxes should remove no longer active Navicache.

_Find geocaches on: gpsgames.org | oc.cz | oc.de On cache page should reflect local options to OCNA. (gpsgames.org, geocaching.com, terracaching.com)

American / British English: unified terms would simplify issue. Select best option. Where there is apparently no cross British / American term that is suitable allow for alternative appropiate term.