Merge old_code with master

vvasuki commented 6 years ago

मित्र,@karthikraman - old_code and master are diverging farther and farther - we might as well be working on entirely different repos. I know you are fond of quick fixes, but if you are seriously interested in getting good jyotisha python code out there, you should spend some time merging these two branches. Else, master lacks all those panchanga fixes you're making in old_code.

karthikraman commented 6 years ago

Apologies, been meaning to ping you on that. I have only been adding festivals, nothing more (as far as I can recall). I had assumed you were in the process of making the old panchanga generation codes talk to the new schema for saving festivals. In the interim, I continued to add festivals, as I received input from various sources... I had been doing this only to keep the panchanga-PDF generation alive...

vvasuki commented 6 years ago

I see logic changes as well as festival data changes - please see https://github.com/sanskrit-coders/jyotisha/pull/22/files . The festival data changes are quite significant. While I have a migrator script ( jyotisha/panchangam/temporal/festival/migrator.py ), since it's been such a long time I had forgotten all about it and had to spend quite some time thinking about it. I don't mind running the script once more, but in the future, it's a hazzle I'd want to avoid - so please continue making even data changes in the master branch's old data jsons till https://github.com/sanskrit-coders/jyotisha/issues/17 is fully resolved.

Can you please merge or manually copy over the changes in https://github.com/sanskrit-coders/jyotisha/pull/22/files to the master branch? For some weird reason git fetch does not seem to retrieve the old-code branch on my computer.

Trying to reconstruct what motivated you to create the old_code branch, I suppose https://github.com/sanskrit-coders/jyotisha/issues/17#issuecomment-360978132 may have been the motivation. But I don't see how the old-code branch scripts resolve this issue (and how any such changes could not have been made in the master branch itself).

karthikraman commented 5 years ago

The manual merge was only half successful. The conflicts I was resolving vanished somehow. I think the primary differences between old code and master are now only in festivals. I want to actually make a new repository for festivals, which can have a consolidated folder with files grouped by topics, from which a migrate.py kind of script populates the new style json, for the efficiencies etc. (if this is not clear, could we please have a brief call sometime?)

karthikraman commented 5 years ago

And this repo can be a module inside panchanga, jyotisha etc... the reason being I'd like to invite more people to contribute to that repo, without touching the panchanga/jyotisha repos...

karthikraman commented 5 years ago

A key reason to have old code was because the daily, monthly and ics panchanga generation codes were broken, and I couldn't debug it. Will take a closer look at the new JSON schema and resolve the errors when I find the time :( - apologies!

vvasuki commented 5 years ago

नमस्काराः कार्तिक।

Let's discuss the " I want to actually make a new repository for festivals, which can have a consolidated folder with files grouped by topics, from which a migrate.py kind of script populates the new style json, for the efficiencies etc. ... And this repo can be a module inside panchanga, jyotisha etc... the reason being I'd like to invite more people to contribute to that repo, without touching the panchanga/jyotisha repos..." - could you open a new thread please? Briefly, I think that's something which merits being taken up after stabilizing the data format issues.

Focusing back on the current problem of closing out old_code.

Is the old_code now finally obsolete? A major obstacle long preventing me from doing many major changes to the master code (including switching over to the new data format) is the fear of further divergence and incompatibility with the old_code branch.
Regarding - "daily, monthly and ics panchanga generation codes were broken, and I couldn't debug it." - that's mysterious because I don't recall any significant change whatsoever in those codes between the two branches. Both were using the same old json files.
I see you have updates files under jyotisha/panchangam/temporal/festival/data/ . Did you use the migrator? You don't need to manual touch those files now. Until https://github.com/sanskrit-coders/jyotisha/issues/17 is fully resolved, jyotisha/panchangam/data/festival_rules.json can remain the canonical data source.

karthikraman commented 5 years ago

Done! Carefully merged all changes into annual.py. Have not yet touched the festivals. Will take a look.

karthikraman commented 5 years ago

The current jsons in master are already modified per your new schema. Hence, I uploaded the updated old_code jsons into a folder called legacy. If you can do the migration (sorry!), master will be in principle completely up-to-date, save for some errors in running the scripts, which you may be able to debug easily. I am looking at them, but haven't figured it yet.

karthikraman commented 5 years ago

I'm getting an error now swisseph.Error: invalid time: 18:30:1874526680.00

karthikraman commented 5 years ago

No precomputed data available. Computing panchangam... Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/scripts/write_daily_panchangam_tex.py", line 246, in <module>
    main()
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/scripts/write_daily_panchangam_tex.py", line 236, in main
    panchangam = scripts.get_panchangam(city=city, year=year, script=script, computeLagnams=computeLagnams)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/scripts/__init__.py", line 32, in get_panchangam
    panchangam.compute_angams(computeLagnams=computeLagnams)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/spatio_temporal/annual.py", line 78, in compute_angams
    ayanamsha_id=self.ayanamsha_id)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/spatio_temporal/daily.py", line 30, in __init__
    self.compute_jd_start()
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/spatio_temporal/daily.py", line 39, in compute_jd_start
    self.julian_day_start = self.city.local_time_to_julian_day(year=year, month=month, day=day, hours=0, minutes=0, seconds=1)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/spatio_temporal/__init__.py", line 96, in local_time_to_julian_day
    julian_dates = swe.utc_to_jd(year_utc, month_utc, day_utc, hours_utc, minutes_utc, seconds_utc, 1)
swisseph.Error: invalid time: 18:30:1874526680.00
error!

vvasuki commented 5 years ago

namaste mitra kartika,

Something seems odd with your setup - panchangam = scripts.get_panchangam(city=city, year=year, script=script, computeLagnams=computeLagnams) workss just fine in my computer (please see https://pastebin.com/1nmQgjt4 ), further this is well tested with tests/spatio_temporal/test_daily.py . Can you please stash away any local changes and retry?

Going forward let's have good pytest tests (I'll move your legacy tests), so that all tests can be automatically run in a single shot and automatic build tools can report errors.

vvasuki commented 5 years ago

mitra kArtika,

Looks like the legacy jsons were really legacy (you timetravelled to the distant past), what you called "new schema" (jyotisha/panchangam/data/festival_rules.json) is actually "old schema" as per https://github.com/sanskrit-coders/jyotisha/issues/17#issuecomment-359190453 ! I did not retain any such migration code, and had to spend quite a few hours matching the fields and writing it - but finally, it is done. Please take a look.

A note for the future-

we should not break each others' code in the first place (as I may or may not have done on your computer as per https://github.com/sanskrit-coders/jyotisha/issues/17#issuecomment-360978132 ) - we should rely on good tests which should pass with each update.
if such breakage does somehow happen, branching off is very suboptimal (even if it is just data) - resulting in net loss of energy, enthusiasm and time (aggregating over both of us).

Soon as you're satisfied, please delete old_code branch so that we may close this issue.

karthikraman commented 5 years ago

Sincere apologies. Instead of branching, I should've just brought up an issue with any breakages. I'll learn :).

Thanks a lot for your efforts!

However, I am still not able to build. Could you remove your json files from ~\Documents\ (the pre-computed ones) and run? (I am trying to run python3 -m jyotisha.panchangam.scripts.write_daily_panchangam_tex which is what I normally use...

Or, is it a problem with Chennai ./gen_daily_cal.sh Chennai 13:05:24 80:16:12 'Asia/Calcutta' 2018 devanagari (from panchangam repo)

Still getting:

No precomputed data available. Computing panchangam... Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/scripts/write_daily_panchangam_tex.py", line 246, in <module>
    main()
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/scripts/write_daily_panchangam_tex.py", line 236, in main
    panchangam = scripts.get_panchangam(city=city, year=year, script=script, computeLagnams=computeLagnams)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/scripts/__init__.py", line 32, in get_panchangam
    panchangam.compute_angams(computeLagnams=computeLagnams)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/spatio_temporal/annual.py", line 79, in compute_angams
    ayanamsha_id=self.ayanamsha_id)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/spatio_temporal/daily.py", line 30, in __init__
    self.compute_jd_start()
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/spatio_temporal/daily.py", line 41, in compute_jd_start
    self.julian_day_start = self.city.local_time_to_julian_day(year=year, month=month, day=day, hours=0, minutes=0, seconds=1)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/spatio_temporal/__init__.py", line 96, in local_time_to_julian_day
    julian_dates = swe.utc_to_jd(year_utc, month_utc, day_utc, hours_utc, minutes_utc, seconds_utc, 1)
swisseph.Error: invalid time: 18:30:274407736.00
error!

vvasuki commented 5 years ago

Thanks! Well, I removed the cached json and it's working ... https://pastebin.com/cKaqjqfV

Did you stash all the local changes?

karthikraman commented 5 years ago

I have no local changes :( Some version issue with swisseph?

I am getting random values for seconds_utc in jyotisha/panchangam/spatio_temporal/__init__.py:95 !

karthikraman commented 5 years ago

In different runs (2017, 12, 30, 18, 30, -298118936) (2017, 12, 30, 18, 30, 330502376)

karthikraman commented 5 years ago

pyswisseph in /usr/local/lib/python3.5/dist-packages/pyswisseph-2.0.0.post2-py3.5-linux-x86_64.egg (2.0.0.post2)

karthikraman commented 5 years ago

I'll do one thing. I'll check on a new computer.

vvasuki commented 5 years ago

Could be version thing.

I remember doing the below, as mentioned in the README:

Install the latest library: sudo pip3 install git+https://github.com/astrorigin/pyswisseph@master -U

karthikraman commented 5 years ago

Thanks! Well, I removed the cached json and it's working ... https://pastebin.com/cKaqjqfV

Did you stash all the local changes?

mitra,

Now it works! But as you have in your pastebin URL, the times are all negative --- some errors in calculation have crept in.. any thoughts?

\caldata{JANUARY}{1}{\sunmonth{धनुः}{17}{}{पौष}{इन्दु}{हेविलम्बी}}
{\sunmoondata{-18:31}{-7:52}{-8:15}{05:21}
{\kalas{-19:41 -15:33 -16:47 -14:18 -8:21 -13:03 -11:19 -9:35 -7:06 -6:42}}}
{\tnykdata{\mbox{\raisebox{-1pt}{\moon[scale=0.8]{13}}\hspace{2pt}शुक्ल-त्रयोदशी\To{}\textsf{-9:28\hspace{2ex}}}}%
{\mbox{रोहिणी\To{}\textsf{-7:53}}}{\mbox{वृषभ-राशी \RIGHTarrow \textsf{04:24}}}%
{\mbox{शुभ\To{}\textsf{-10:04\hspace{2ex}}}}%
{\mbox{तैतिल\To{}\textsf{-9:28\hspace{2ex}}}\mbox{गरज\To{}\textsf{01:38\hspace{2ex}}}}{\tiny }
}
{\rygdata{-17:56---15:21}{-14:46---12:11}{-11:36---9:01}}
{}
{Mon}

CORRECT:

\caldata{JANUARY}{1}{\sunmonth{धनुः}{17}{}{पौष}{इन्दु}{हेविलम्बी}}
{\sunmoondata{06:32}{17:52}{17:16}{06:25(+1)}
{\kalas{05:41 09:33 08:48 10:19 16:21 11:04 13:20 15:36 17:07 18:43}}}
{\tnykdata{\mbox{\raisebox{-1pt}{\moon[scale=0.8]{14}}\hspace{2pt}शुक्ल-चतुर्दशी\To{}\textsf{11:44\hspace{2ex}}}}%
{\mbox{मृगशीर्ष\To{}\textsf{14:53}}}{\mbox{मिथुन-राशी}}%
{\mbox{शुक्ल\To{}\textsf{09:47\hspace{2ex}}}\mbox{ब्रह्म\To{}\textsf{05:23(+1)\hspace{2ex}}}}%
{\mbox{वणिज\To{}\textsf{11:44\hspace{2ex}}}\mbox{भद्र\To{}\textsf{21:49\hspace{2ex}}}}{\tiny }
}
{\rygdata{07:57--09:22}{10:47--12:12}{13:37--15:02}}
{अन्धकासुर-वधः\eventsep काञ्ची ८ जगद्गुरु श्री~कैवल्यानन्दयोगेन्द्र सरस्वती आराधना~\#{१९८९}\eventsep वेङ्कटाचले पौर्णमी~गरुड-सेवा}

vvasuki commented 5 years ago

Fixed - that was a bug I introduced yesterday. I had meant to add a test to compare the daily calendar output with the files you emailed, but I didn't get to yet - will add it today.

karthikraman commented 5 years ago

Super. I diffed with the file I sent yesterday and there are minor round-off changes (very few) in ending times of angas, probably because of swisseph version change. I think this can now be closed!

Thanks so much! And apologies for increasing your work so much!!

Happy Deepavali!

vvasuki commented 5 years ago

Great - thanks for checking!

Deleted old-code branch.

karthikraman commented 5 years ago

Oops, I hadn't checked the ics file generation. That seems to have a minor bug:

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/scripts/ics.py", line 268, in <module>
    main()
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/scripts/ics.py", line 263, in main
    ics_calendar = compute_calendar(panchangam)
  File "/home/karthik/GitHub/jyotisha/jyotisha/panchangam/scripts/ics.py", line 201, in compute_calendar
    jyotisha.custom_transliteration.tr(festival_rules[stext].get('shlokas'), panchangam.script, False) + '\n\n'
  File "/home/karthik/GitHub/jyotisha/jyotisha/custom_transliteration.py", line 39, in tr
    text = text.replace('~', '##~##')  # Simple fix to prevent transliteration of ~
AttributeError: 'NoneType' object has no attribute 'replace'
error!

I think it's returning a list instead of a string perhaps... I'll look more closely too..

vvasuki commented 5 years ago

Fixed.

karthikraman commented 5 years ago

There's no error but my ICS file is empty...

vvasuki commented 5 years ago

Please see attached output - perhaps you're looking at the wrong path (I changed the output path to go to ~/Documents). Chennai-2018-devanagari.txt

jyotisham / jyotisha

Merge old_code with master #21