crbyxwpzfl / mini

sync to mini macOS

0 stars 0 forks source link

refactor pullreadlist py #18

Closed crbyxwpzfl closed 2 years ago

crbyxwpzfl commented 2 years ago

add aria2c support

[x] install location see grey marked tag
[x] aria2c --enable-rpc --rpc-listen-all --on-download-complete=/Users/mini/Desktop/hook.sh --daemon true start aria in daemon mode with download completion hook
[x] It will download to the working directory of the RPC server. If you want to change it start the server with --dir option or set it for each file - instead of {"out":"test.jpg"} in the JSON you send use {"out":"test.jpg","dir":"path_to_folder"}
[x] data = {'jsonrpc': '2.0', 'id': 'qwer', 'method': 'aria2.addUri','params':[[link],{"out":"penis.webm"}]} data to send to aria2 daemon to add link and uses option out for filename
[x] print(requests.post('http://localhost:6800/jsonrpc', json=data) .content) sends call to aria daemon on local host and prints response
[x] for download completion hook use py script with shebang #!/usr/bin/env python3 at top of script info link
[x] shutdown via rpc data = {'jsonrpc': '2.0', 'id': 'qwer', 'method': 'aria2.shutdown'} print(requests.post('http://localhost:6800/jsonrpc', json=data) .content)
[x] How are dublicats handled eg adding the same link multiple times to daemon Info
[x] get status of download and count of downloads data = {'jsonrpc': '2.0', 'id': 'qwer', 'method': 'aria2.tellActive','params':[['gid','status','dir','files']]} data = {'jsonrpc': '2.0', 'id': 'qwer', 'method': 'aria2.tellStopped','params':[0,3,['gid','status','dir','files']]}
```
import requests
import json
```

remember to set right offset in params

data = {'jsonrpc': '2.0', 'id': 'qwer', 'method': 'aria2.tellStopped','params':[0,20,['gid','status','files', 'dir','errorMessage']]}

r = requests.post('http://localhost:6800/jsonrpc', json=data) print(type(r.content)) parsed = json.loads(r.content)

resultlist = parsed['result'] print(type(resultlist))

for item in resultlist: print("") print(item['errorMessage']) print(item['status'], item['dir'],item['files'][0]['path']) print("")



- [x] see if aria is running `killall -s aria2c` 

- [x] test how restarts of aria are handled
and how restarts in general are handled
`aria2c --enable-rpc --rpc-listen-all --daemon=true --auto-file-renaming=false --allow-overwrite=false --seed-time=0 --save-session=/Users/mini/Desktop/ariasave.txt --input-file=/Users/mini/Desktop/ariasave.txt ` 

- [x] #40
`https://video3.xhcdn.com/key=Rfhyj7avf3umZ8C9p-DKXA,end=1652040000/data=89.247.207.197-dvp/media=hls4/006/059/131/720p.h264.mp4.m3u8`

crbyxwpzfl commented 2 years ago

switch to yt-dlp

[x] for instagramm support
[x] select best is different form select best audio / best video
[x] python3 /Users/mini/Downloads/youtube-dl\ 2/youtube-dl -g -f best --get-filename 'url' try to use embedded variant instead of above cmd to retrieve extracted url to pass to aria

import yt_dlp

# Selector works with url from 
# yt            'https://www.youtube.com/watch?v=BaW_jenozKc'
# redgif        'https://www.redgifs.com/watch/mealyshockingwildebeest'
# reddit/imigur 'https://www.reddit.com/r/nsfw/comments/uk8lj8/rain_and_smile/'
# ig            'https://www.instagram.com/reel/CbVo41uJX19/'
# imigur        'https://imgur.com/bqRKKq5'
# xh            'https://deu.xhamster.com/videos/attractive-teen-alaina-dawson-makes-her-stepdad-cum-on-her-6059131'

URLS = 'https://www.instagram.com/reel/CbVo41uJX19/'

ydl_opts = {'formats': 'bestvideo*,bestaudio'} # get best vid no matter the audio + get best audio

with yt_dlp.YoutubeDL(ydl_opts) as ydl:
    test = ydl.extract_info(URLS, download=False)

    #for key in test.keys(): #print all keys of dict
    #    print (key)

    print(ydl.list_formats(test)) #lists formats

    if test.get('requested_formats'):
        reqf = test['requested_formats'] 
        for f in reqf:
            print(ydl.prepare_filename(f, outtmpl='filename-vc:%(vcodec)s-ac:%(acodec)s.%(ext)s'))
            print(f['format'],f['format_id'],f['url'])
            print("")
    else:
        print(ydl.prepare_filename(test, outtmpl='filename-vc:%(vcodec)s-ac:%(acodec)s.%(ext)s'))
        print(test['format'],test['format_id'],test['url'])

crbyxwpzfl commented 2 years ago

ffmpeg redo

ffmpeg -i "input.mkv" -metadata title= -map 0 -vcodec copy -acodec copy -scodec "mov_text" -ac 8 ultimate-test.mp4 --> carries all subs --> fixes audio for apple tv --> no reencoding just remuxing and perhaps audio downmixing but shud carry audio up to 7.1
ffmpeg -i audio.mp3 -i video.mp4 output.mp4 combine best audio and best video form yt-dl into one file perhaps remuxing / reencoding is needed for apple quick time support

# single subtitle with MP4(for MOV : change mov_text to srt)
$ ffmpeg -i VIDEO.mp4 -i SUBTITLE.srt -c:v copy -c:a copy -c:s mov_text OUTPUT.mp4
$ ffmpeg -i VIDEO.mov -i SUBTITLE.srt -c:v copy -c:a copy -c:s srt OUTPUT.mov

# multiple subtitles with MP4(for MOV : change mov_text to srt)
$ ffmpeg -i VIDEO.mp4 -i KOREAN.srt -i ENGLISH.srt \
  -c:v copy -c:a copy -c:s mov_text \
  -map 0:v -map 0:a -map 1 -map 2 \
  -metadata:s:s:0 language=kor -metadata:s:s:1 language=eng \
  OUTPUT.mp4

crbyxwpzfl commented 2 years ago

others

evtl inclode favicons and chang favicon on click on index html
include motion calls and various iMessages on updates
consider running script on startup
add ssh private to new privatespy for pullreadlist
1 automate vpn
2 include surce.py functionality and integrate into homebridge
3 sanitise file names and perhaps add metadata to files
4 cleanup tmps folder

crbyxwpzfl commented 2 years ago

[x] use multi opening insted of aria daemon for ytdl https://stackoverflow.com/questions/9474859/how-do-i-count-specific-processes-on-mac-os-x
[x] shebangs
```
import subprocess
import time
```

while True: i = 0

cmd = ['killall', '-s', 'ping']
process = subprocess.Popen(cmd, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in process.stdout:
    #print(line)
    i = i + 1

print("pings running", i)

if i < 4:
    #cmd2 = ['osascript', '-e', 'tell app "Terminal" to do script "/Users/mini/Desktop/shebangs.py world"']
    cmd2 = ['osascript', '-e', 'tell app "Terminal" to do script "ping google.com -i 10"']
    test = subprocess.run(cmd2, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    #print(test)
    print("started ping")

time.sleep(2)


```py
#chmod +x /current/file
#!/usr/bin/env python3

import time
import sys

while True:
    print("hi", sys.argv[0], sys.argv[1]) 
    time.sleep(2)

crbyxwpzfl commented 2 years ago

[x] propper readinglist parsing


import sys
import subprocess
import os
import plistlib

def sub(cmd): process = subprocess.Popen(cmd, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) for line in process.stdout: print(line)

bookmarksxml = '/Users/mini/Desktop/SafariBookmarks.xml' #where to export bookmarks to bookmarksplist = os.path.join(os.environ.get('HOME'), 'Library', 'Safari', 'Bookmarks.plist') sub(['plutil', '-convert', 'xml1', '-o', bookmarksxml, bookmarksplist]) #convert bookmark plist to xml

plist = plistlib.load(open(bookmarksplist, 'rb')) for child in plist['Children']: if child.get('Title', None) == 'com.apple.ReadingList': for item in child['Children']: print(item['URLString'])


```py
import os
import plistlib
import requests

INPUT_FILE = os.path.join(os.environ['HOME'], 'Library/Safari/Bookmarks.plist')
OUTPUT_FILE = 'readinglist.txt'

# Load and parse the Bookmarks file
with open(INPUT_FILE, 'rb') as plist_file:
    plist = plistlib.load(plist_file)

# Look for the child node which contains the Reading List data.
# There should only be one Reading List item
children = plist['Children']
for child in children:
    if child.get('Title', None) == 'com.apple.ReadingList':
        reading_list = child

# Extract the bookmarks
bookmarks = reading_list['Children']

for bookmark in bookmarks:
    url = bookmark['URLString']
    try:
        text = bookmark['ReadingList']['PreviewText']
    except:
        text = ""
    title = bookmark['URIDictionary']['title']
    ret = requests.head(url)
    print("%s, %d, %s, %s" % (url, ret.status_code, title, text))

crbyxwpzfl commented 2 years ago

[x] get vpn status + location
[x] edit spinala/index.html accordingly
[x] push changes to github

optional

[x] ommit unused characteristics form dict
[ ] find out why truncating is neccessary
[x] wirte beter parser for nordvpn output
```
import sys
import os
```

import subprocess

def Get(): print(dict.get(sys.argv[3].strip("''") , 1)) sys.exit()

def My(): cmd = ['ssh', 'pi@192.168.2.1', '-i', '/Users/mini/Downloads/private/gscheit-openssh-privat', 'nordvpn', 'status'] r = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) if "Disconnected" in str(r.stdout): print("vpn off") file = "/Users/mini/Desktop/indexcopy.html" with open(file, 'r+') as fd: contents = fd.readlines() contents[6] = "#de {fill: #fc4444;} / set de here if vpn off /\n" contents[7] = "#de:hover {stroke: #fc4444; stroke-width: 4; fill: #fc4444;} / set de here if vpn off /\n" fd.seek(0) fd.writelines(contents) fd .truncate()

if "Connected" in str(r.stdout):
    print("vpn on")
    loc = str(r.stdout)[55:57] # very dumn parsing but output is not serializable subject ot cahnge
    print(loc)

    file = "/Users/mini/Desktop/indexcopy.html"
    with open(file, 'r+') as fd: 
        contents = fd.readlines()
        contents[6] = f"#{loc} {{fill: #5cf287;}}  /* set de here if vpn off */\n"
        contents[7] = "\n"
        fd.seek(0)
        fd.writelines(contents)
        fd .truncate()

def clone(): sshpriv = '/Users/mini/Downloads/private/gscheit-openssh-privat' dir = '/Users/mini/Desktop/' dir2 = '/Users/mini/Desktop/spinala' cmd = ['git', '-C', dir,'-c', f"core.sshCommand=\"\"ssh -i {sshpriv}\"\"", 'clone', 'git@github.com:crbyxwpzfl/spinala.git'] r = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) print(r.stdout) cmd = ['git', '-C', dir2, '-c', f"core.sshCommand=\"\"ssh -i {sshpriv}\"\"", 'pull'] r = subprocess.run(cmd,text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) print(r.stdout)

#if all up to date
#change file now
#git commit -am "site update"
#git -c core.sshCommand="ssh -i /Users/mini/Downloads/private/gscheit-openssh-privat" push

cmd = ['git', '-C', dir2, 'commit', '-am', '"site update"'] #does not picup on new created files
r = subprocess.run(cmd,text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
print(r.stdout)
cmd = ['git', '-C', dir2, '-c', f"core.sshCommand=\"\"ssh -i {sshpriv}\"\"", 'push']
r = subprocess.run(cmd,text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
print(r.stdout)

dict = {'clone': clone, 'my': My, 'Get': Get, 'CurrentRelativeHumidity': 80, 'StatusActive': 1, 'StatusFault': 0, 'StatusTampered': 0, 'StatusLowBattery': 0} #bat und fault keine anzeige dict.get(sys.argv[1].strip("''"), sys.exit)() # call 'Get' or sys exit()

crbyxwpzfl commented 2 years ago

just to save some aria requests

        get( {'jsonrpc': '2.0', 'id': 'mini', 'method': 'aria2.tellStopped','params':[0,20,['gid','status','files', 'dir','errorMessage']]} )# get info about arias queue TODO set right offset in params
        get( {'jsonrpc': '2.0', 'id': 'qwer', 'method': 'aria2.tellActive','params':[['gid','status','dir','files']]} )
        get( {'jsonrpc': '2.0', 'id': 'qwer', 'method': 'aria2.tellWaiting','params':[0,3,['gid','status','dir','files']]} )

        get({'jsonrpc': '2.0', 'id': 'mini', 'method': 'aria2.getGlobalStat'})

        requests.post('http://localhost:6800/jsonrpc', json={'jsonrpc': '2.0', 'id': 'mini', 'method': 'aria2.purgeDownloadResult'}, verify=False, timeout=2)
        requests.post('http://localhost:6800/jsonrpc', json={'jsonrpc': '2.0', 'id': 'mini', 'method': 'aria2.getGlobalStat'}, verify=False, timeout=2)

data = {'jsonrpc':'2.0', 'id':'mini', 'method':'system.multicall', 'params':[[{'methodName':'aria2.getGlobalStat'}, {'methodName': 'aria2.tellStopped', 'params':[0,20,['status', 'files', 'errorMessage']]}]]}
print(json.loads(requests.post('http://localhost:6800/jsonrpc', json=data, verify=False, timeout=2).content)['result'])

         aria2.getGlobalStat([secret])¶
          aria2.purgeDownloadResult()

[{'jsonrpc': '2.0', 'id': 'mini', 'method': 'aria2.getGlobalStat'}, {'jsonrpc': '2.0', 'id': 'mini', }]

 {'jsonrpc':'2.0', 'id':'qwer', 'method':'system.multicall', 'params':[[{'method': 'aria2.getGlobalStat'}, {'method': 'aria2.tellStopped','params':[0,20,['status', 'files', 'errorMessage']] }]]}

{'jsonrpc':'2.0', 'id':'mini', 'method':'system.multicall', 'params':[[{'methodName':'aria2.getGlobalStat'}, {'methodName': 'aria2.tellStopped', 'params':[0,20,['status', 'files', 'errorMessage']] } ] ] }

crbyxwpzfl commented 2 years ago

OKOK essential stuff to be added

[x] make dlp always putout mp4 and mp3
[x] make aria mkvs auto remux to mp4 and auto rename according to message
[x] re work messages on vpn change
[x] change and verify country codes form ipinfo to match country codes from vpn

crbyxwpzfl commented 2 years ago

some thoughts

dump homebridge and setup new server all together ideally webserver
re implement script into homebridge wtih simpler accesorie no uneccessary fancyness i never look at