search

jarun / buku

:bookmark: Personal mini-web in text
GNU General Public License v3.0
6.53k stars 294 forks source link

ToDo list #484

Open jarun opened 3 years ago

jarun commented 3 years ago

Continued from #343.

Notes

Please check the open issues for an area to contribute. bukuserver has several open features requests.

The list below is a growing one. While suggesting new features please consider contributing to Buku. The code is intentionally kept simple and easy to understand with comments. We'll be happy to assist any new contributor. We need your help!

Some of the fresh-baked features may not have been released yet. Grab the master branch for those.

Cooking

Identified tasks

rachmadaniHaryono commented 3 years ago

i just noticed that there is possibility to track user with favicon based on this https://github.com/jonasstrehle/supercookie

i may update bukuserver later to set favicon default to false.

jarun commented 3 years ago

Sure thing!

zz5zz commented 3 years ago

The bukuserver is an awesome frontend for Buku!

Some new functionality:

jarun commented 3 years ago

@zz5zz are you interested in raising PRs?

@rachmadaniHaryono please review.

rachmadaniHaryono commented 3 years ago

@zz5zz

Add a Dark mode and possibility to turn on & off (create a custom CSS) Bootstrap should be updated to minimum "bootstrap 3.4.1" to remove the known vulnerabilities (https://snyk.io/vuln/npm:jquery?lh=3.3.1) jQuery should be updated to minimum "jquery 3.5.0" to remove the known vulnerabilities (https://snyk.io/vuln/npm:bootstrap?lh=3.3.5)

these all depend on flask-admin, but i do hope bukuserver can be updated to use latest flask before updating flask admin

Add group / category to be used together with Tags to create a hierarchy when a lot of bookmarks have been added . That way you can as an example create a group called Newspapers with tags like electronics, legal, sports, economics etc.

bukuserver depend on buku to handle tag so no group and category. but it is actually possible if bukuserver can parser tag and handle part of tag as category/group, for example programming: python tag will be under programming category group.

Add year & month when added for both navigation purpose and stats over time (how many bookmarks was created in X)

buku don't record created date

When modal opens, should select the first input field by default

this depend on flask admin and should be tested when flask admin updated

Add mousetrap.js to make keyboard shortcuts available like example CTRL+SHIFT+A to create a new bookmark

that seem to be nice addition, but it should be noted that bukuserver should handle external javascript dependency before adding new one . see also #307

zz5zz commented 3 years ago

@zz5zz are you interested in raising PRs?

@rachmadaniHaryono please review.

I'll try to contribute by doing a PR. Very long time since I handled python so need to brush up a bit.

jarun commented 3 years ago

Please take your time. Feel free to reach out for any help.

RichardConroy commented 3 years ago

What would it take to integrate the Buku database with Mac Spotlight search ? This strikes me as being a separate project, but its hard to know where to get started with it.

jarun commented 3 years ago

Sorry, I am not a Mac user, no ideas.

froschdesign commented 3 years ago

@RichardConroy You can use Alfred with a custom workflow. For the script you can use Python (or what you like) to execute the buku CLI command, the filtering of the bookmarks can be done by Alfred.

GreenLunar commented 2 years ago

Please add: Utilizing Bookie https://github.com/bookieio/Bookie

rachmadaniHaryono commented 2 years ago

i create #546 but after few new bukuserver issue i decided to convert it to draft and merge possible small issue into this pr

unfortunately i can't start weekend but hopefully next few days i can work on it

and maybe if possible upgrade bukusserver to newer flask version

Remiol commented 2 years ago

Buku is a great bookmark manager. I use it daily. I wish some of the related projects also work on windows like the bukubrow server and Oil. Search-as-you-type cli front-end. Maybe also an interface to gvim. Anyway, thank you for the good work you've done.

GreenLunar commented 2 years ago

XMPP bot (also other IM platforms that provide API for bots)

rachmadaniHaryono commented 2 years ago

this is current pyproject.toml for buku

[tool.poetry]
name = "buku"
version = "4.6"
description = "Bookmark manager like a text-based mini-web."
authors = ["Arun Prakash Jana <engineerarun@gmail.com>"]
license = "GPLv3"
keywords = ["cli", "bookmarks", "tag", "utility"]
classifiers = [
  "Development Status :: 5 - Production/Stable",
  "Environment :: Console",
  "Intended Audience :: Developers",
  "Intended Audience :: End Users/Desktop",
  "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
  "Natural Language :: English",
  "Operating System :: OS Independent",
  "Programming Language :: Python :: 3 :: Only",
  "Programming Language :: Python :: 3",
  "Programming Language :: Python :: 3.6",
  "Programming Language :: Python :: 3.7",
  "Programming Language :: Python :: 3.8",
  "Topic :: Internet :: WWW/HTTP :: Indexing/Search",
  "Topic :: Utilities",
]
homepage = "https://github.com/jarun/buku"
readme = "README.md"

[tool.poetry.dependencies]
python = ">=3.6,<4"
beautifulsoup4 = ">=4.4.1"
cryptography = ">=1.2.3"
html5lib = ">=1.0.1"
urllib3 = ">=1.23"
certifi = {version = "^2021.10.8", extras = ["ca-certificates"]}
appdirs = {version = ">=1.4.3", extras = ["server"]}
arrow = {version = ">=0.12.1", extras = ["server"]}
cffi = {version = ">=1.9.1", extras = ["server"]}
click = {version = ">=7.0", extras = ["server"]}
Flask-API = {version = ">=0.6.9", extras = ["server"]}
Flask-Bootstrap = {version = ">=3.3.7.1", extras = ["server"]}
flask-paginate = {version = ">=0.5.1", extras = ["server"]}
flask-reverse-proxy-fix = {version = ">=0.2.1", extras = ["server"]}
Flask-WTF = {version = ">=0.14.2", extras = ["server"]}
Flask = {version = ">=1.0.2,<2.0", extras = ["server"]}
idna = {version = ">=2.5", extras = ["server"]}
itsdangerous = {version = ">=0.24", extras = ["server"]}
Jinja2 = {version = ">=2.10.1", extras = ["server"]}
MarkupSafe = {version = ">=1.0", extras = ["server"]}
packaging = {version = ">=16.8", extras = ["server"]}
pyasn1 = {version = ">=0.2.3", extras = ["server"]}
pycparser = {version = ">=2.17", extras = ["server"]}
requests = {version = ">=2.21.0", extras = ["server"]}
six = {version = ">=1.10.0", extras = ["server"]}
Werkzeug = {version = ">=0.11.15,<0.15.0", extras = ["server"]}
flask-admin = {url = "https://github.com/flask-admin/flask-admin/archive/refs/heads/master.zip", extras=["server"]}
twine = {version = "^3.7.1", extras = ["packaging"]}

[tool.poetry.dev-dependencies]
attrs = ">=17.4.0"
flake8 = ">=3.4.1"
hypothesis = ">=6.0.0"
mypy-extensions = "0.4.1"
py = ">=1.5.0"
pylint = ">=1.7.2"
pytest-cov = "^3.0.0"
pytest-vcr = ">=1.0.2"
pytest = ">=6.2.1"
PyYAML = ">=4.2b1"
setuptools = ">=41.0.1"
vcrpy = ">=1.13.0"
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
[tool.poetry.scripts]
buku = 'buku:main'
bukuserver='bukuserver.server:cli'

the biggest change should be werkzeug for bukuserver 

Werkzeug = {version = ">=0.11.15,<0.15.0", extras = ["server"]}

this is because flask_api require werkzeug._compat which only exist before v0.15.0

so therefore bukuserver packages that need to be upgraded

i'm still waiting pr for flask-reverse-proxy-fix

https://github.com/antarctica/flask-reverse-proxy-fix/pull/1

if i finished with current bukuserver bugfix pr, i will fork the project and create new version

related issue

https://github.com/jarun/buku/issues/449

why using poetry?

from their website

while there is guideline, there is no entry about virtualenv and handling package requirements

predicted issue

to handle pypi release when this is accepted, maybe release alpha/beta/rc version before final version

there are also several tool that i want to add to project

with any of this tool is accepted, guideline on wiki should be changed as well

python 3.6 just ended on december 2021

https://endoflife.date/python

e:

follow nep 29 https://numpy.org/neps/nep-0029-deprecation_policy.html

idea: if poetry is accepted, post about recruiting new beginner developer for help on doc, test, and program

most of the time, beginner developer can see what is needed or what is not clear on buku

minimal requirement would be just understand poetry and git

e2:

add doc for using pipx to install buku

rachmadaniHaryono commented 2 years ago

@jarun, sorry i think i push a commit accidentally. c09ecfbc2745bcc7f2e70942975bfceb0536075a

can you revert it? i think i can do it, but it is better if you do it.

sorry for this

rachmadaniHaryono commented 2 years ago

unrelated to above post

but in this issue i thought we could use yaml for issue form

https://github.com/jarun/buku/issues/558#issuecomment-1011482388

imo i think we can update issue template so user have to fill buku version

https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#creating-issue-forms

but i miss the notification on top of that page

Note: Issue forms are currently in beta for public repositories on GitHub.com only.

i thought buku should update its template because when you go to https://github.com/jarun/buku/blob/master/.github/ISSUE_TEMPLATE.md there is always this notification

You are using an old version of issue templates. Please update to the new issue template workflow. Learn more

so i create new issue template and feature request template from github

results

https://github.com/rachmadaniHaryono/Buku/issues/new/choose

https://github.com/rachmadaniHaryono/Buku/blob/master/.github/ISSUE_TEMPLATE/bug_report.md

---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.
Before opening an issue, please try to reproduce on [the latest development version](https://github.com/jarun/Buku#from-source) first. The bug you noticed might have already been fixed.
If we need more information and there is no communication from the bug reporter within 7 days from the date of request, we will close the issue. If you have relevant information, resume discussion any time.

**To Reproduce**
Steps to reproduce the behavior:
1. ...
2. ...
3. ...
4. See error

Debug logs using the `-g` option

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
 - Buku version: [e.g. 4.6, or commit hash]
 - OS: [e.g. windows, mac, linux, ubuntu]
 - Python version [e.g. 3.9, 3.10]
 - Terminal emulator and shell [e.g. "alacritty, zsh" or "gnome-terminal, fish"]
 - `locale` output, if relevant [e.g. LANG=en_US.UTF-8]

It's a good idea to set your locale to UFT-8. Please refer to [Buku #131](https://github.com/jarun/Buku/issues/30).

**Additional context**
Add any other context about the problem here.

https://github.com/rachmadaniHaryono/Buku/blob/master/.github/ISSUE_TEMPLATE/feature_request.md

---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
Please consider contributing the feature back to `Buku` yourself. Feel free to discuss. We are more than happy to help.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.

looking at current template, i don't know if it is worth it to change it

rachmadaniHaryono commented 2 years ago

i tried to use poetry for buku not haven't found better configuration yet.

there is still issue to use it without copy buku to buku.py

i also kinda wary(?) about this because this should discussed with other packager

i also tried to create github page with sphinx on current doc, but it is harder

another possibility is to replace it with mkdocs, but that is last option

i am more familiarized with mkdocs for now, so maybe i will try it (and replace sphinx)

pre commit update pr is accepted, so it should be easier for contributor to automate some task

but contributing.md is not yet updated for this

unfortunately i don't give buku time for now, so no fixes for bukuserver for now

apraga commented 2 years ago

Is there a built-in way to add a new tag from the prompt ? From what I understand, g only works with existing tag. Thanks !

jarun commented 2 years ago

Use w to edit a bookmark and add a tag. You can add it to more bookmarks after that.

apraga commented 2 years ago

Right, that’s what I’ve been doing. Here are 2 suggestions if other people are interested :

jarun commented 2 years ago

buku design doesn't allow having a tag that's not linked to at least one bookmark.

rachmadaniHaryono commented 1 year ago

current state for bukuserver before new release

bug (important)

feature (lower priority, should add to todo list)

e: 606 already done

rachmadaniHaryono commented 1 year ago

…Speaking of running tests locally – one of test_network_handler_with_url runs (corresponding to a Google search query) always fails on my machine due to Google server responding with HTTP 429 Too Many Requests (I get the same response when sending the request with wget).

Not sure why this happens (perhaps it's related to me using a VPN for internet access?), but it certainly results in guaranteed fails every time I run unit tests locally. (…Does it really count as a unit test if the network requests are actually being sent? Normally you'd expect the requests themselves to be mocked or at least extracted to external functionality, in part to avoid such factors from affecting the outcome – after all, I doubt that the goal is to test specifically whether the code can communicate with Google servers.)

Does it really count as a unit test if the network requests are actually being sent?

@LeXofLeviafan

but it kinda also show that this function will not result the same thing on different computer, so it kinda work

but for consistency it is actually bad unit test

several options:

  1. have second alternative result instead of fail the test immediately
  2. cache the result, something similar to https://github.com/oprypin/pytest-golden
  3. mock the result

e: related https://github.com/jarun/buku/pull/443

LeXofLeviafan commented 1 year ago

Unit tests are meant to test the code itself – whether the logic works correctly. And for a test to show that it "will not result in the same thing on different computer", it needs to be purposefully run on multiple machines (…also such a thing normally indicates that either the test or the code is broken :sweat_smile:)

The reason it fails is purely incidental and completely external (i.e. this particular request has been blocked by Google for specific IPs or something like that) which has nothing to do with what the code is being tested for, really (unless the purpose is to test how various servers on the internet handle repeating requests or how their DDoS detection works).

…The caching thing looks like the way to go (provided the cache is actually included in the repo), except what you'd want is to cache the network requests instead of tests outputs. Ideally you'd want to generate such a cache file once (and never really need to modify it afterwards), and then mock the network environment in the test by providing stored response instead of live network feedback (or simply separate I/O from logic in the code and test the logic only). It can even be archived/compressed (a tarball?) to avoid having many small files or a large one in the repo (which seems to be the reason why #443 got rejected).

rachmadaniHaryono commented 1 year ago

current issue with no active pr

e: i'm sorry network cache pr is late, hopefully i can start tomorrow

e2: remove 644

LeXofLeviafan commented 1 year ago

…#644 does have a pull request though :-)

rachmadaniHaryono commented 1 year ago

@LeXofLeviafan you are correct, i will remove that

LeXofLeviafan commented 1 year ago

current issue with no active pr

I just realised that #648 appears to be missing from the list.

rachmadaniHaryono commented 1 year ago

added to refactor, thanks @LeXofLeviafan

LeXofLeviafan commented 1 year ago

API changes in #660:

LeXofLeviafan commented 1 year ago

i just noticed that there is possibility to track user with favicon based on this https://github.com/jonasstrehle/supercookie

For the sake of completeness, I'll mention it here as well: the favicons in the app are downloaded from Google server and are treated as regular images, so their sites are not even aware of the user fetching them, let alone having means to make them individual per-user or persist through cache cleaning.

LeXofLeviafan commented 1 year ago

Suggestion: release this fork on PyPI (seeing as the original repo is stale), and change the [reverse_proxy] dependency accordingly (possibly reintegrating it back into the [server] deplist).

sjehuda commented 5 months ago

support offline bookmark addition

This seems to be done with argument --offline.

LeXofLeviafan commented 2 months ago

@jarun I've listed all pull-requests as items in the "Cooking" list (grouping when necessary); some might be considered too minor to list in the changelog though (at least individually). Added the in-progress stuff as well (as unchecked items).

…Also, you may want check off some of the "Identified tasks" list items; I think some of them at least are done by now.