qlands / elasticfeeds

A Python library for managing feeds using ElasticSearch
Other
19 stars 4 forks source link

sanity check on following / feeds by user / unpacking tests #7

Closed wweevv-johndpope closed 5 months ago

wweevv-johndpope commented 1 year ago

FYI - worth a read regarding elasticsearch 8 + licenses. https://aws.amazon.com/what-is/opensearch/

I'm at a point where I don't know how / why things work. or how I think things should work - are not aligning.

I have some superficial questions on code / tests. here goes - thanks in advance for any insights.

Question is) where is edoquiros in test_00_manager? https://github.com/qlands/elasticfeeds/blob/master/elasticfeeds/tests/test_00_manager.py#L67C36-L67C45

because you're using "cquiros" it is describing linkedActivity / activity_id / Link / cquiros - can you help unpack this?

    # Creates a linked activity
   # tst_linked_activity = LinkedActivity("cquiros") ## q) is this supposed to be edoquiros? 
    tst_linked_activity = LinkedActivity(activity_id=str(uuid.uuid4())) # q) if it is really "cquiros" - then it seems it follows itself.
    tst_linked_activity.activity_class = "actor"
    tst_linked_activity.activity_type = "person"

    # Creates a link

    tst_link = Link(actor_id="cquiros",linked_activity=tst_linked_activity)# UPDATE ) digging through code - maybe it's better to spell this  actor_idout....
    tst_link.linked = now
    tst_link.link_type = "follow"
    tst_link.link_weight = 1
    tst_link.extra = {"some_extra_data": "test"}

    # Adds the network link
    tst_manager.add_network_link(tst_link)

    # Carlos follow Eduardo. Test of convenience function Q) is this Carlos follows back Eduardo?
    tst_manager.follow("cquiros", "edoquiros", now) # Q) is there no referential integrity? where does edoquiros come from?

BACKGROUND

I successfully create the indexes


    tst_manager = Manager(
        feed_index = "testfeeds",
        network_index = "testnetwork",
        byo_connection = True
    )
    tst_manager.assign_connection(opensearch)

In another process / importing liked youtube videos

this successfully adds the activity / it's basically - a user "Blabla" liked a video - and the extra field contains youtube content.

✅ - this is working

  tst_actor = Actor(actor_id, "person",extra={"url":"imagelink"})
    # Creates an object

    # video_id = dict['video_id']
    # title = dict['title']
    # description = dict['description']
    # url = dict['url']    
    tst_object = Object(video_id, "video")
    # Creates an Activity
    tst_activity = Activity(
        activity_type="liked", activity_actor=tst_actor,activity_object=tst_object, published=now + datetime.timedelta(minutes=12),extra={"url":video_url,"description":description,"title":title,"vid":vid}
    )
    # Adds the activity
    tst_manager.add_activity_feed(tst_activity)

and I get the following json using this query

GET _search
{
  "query": {
    "match_all": {}
  }
}

✅ - this is working


      {
        "_index": "testfeeds",
        "_id": "ea2839ae-8cdf-4be8-b928-7914941385d9",
        "_score": 1,
        "_source": {
          "published": "2023-08-23T08:17:33.350965",
          "published_date": "2023-08-23",
          "published_time": "08:17:33",
          "published_year": 2023,
          "published_month": 8,
          "actor": {
            "id": "blabla",
            "type": "person",
            "extra": {
              "url": "imagelink"
            }
          },
          "type": "liked",
          "object": {
            "id": "OuzUyrcvTg4",
            "type": "video"
          },
          "extra": {
            "url": "https://www.youtube.com/watch?v=OuzUyrcvTg4",
            "description": """The impossibblablalblalblba.""",
            "title": "Pilot Tries Impossible Turn!",
            "vid": {
              "kind": "youtube#video",
              "etag": "E3ofbOEgXPR0rsC3xav4H2b88Zk",
              "id": "OuzUyrcvTg4",
              "snippet": {
                "publishedAt": "2023-08-05T15:03:39Z",
                "channelId": "UCBeZYVlqOeSSlrBSXl4aTig",
                "title": "Pilot Tries Impossible Turn!",
                "description": """The imposblablablalballab.""",
                "thumbnails": {
                  "default": {
                    "url": "https://i.ytimg.com/vi/OuzUyrcvTg4/default.jpg",
                    "width": 120,
                    "height": 90
                  },
                  "medium": {
                    "url": "https://i.ytimg.com/vi/OuzUyrcvTg4/mqdefault.jpg",
                    "width": 320,
                    "height": 180
                  },
                  "high": {
                    "url": "https://i.ytimg.com/vi/OuzUyrcvTg4/hqdefault.jpg",
                    "width": 480,
                    "height": 360
                  },
                  "standard": {
                    "url": "https://i.ytimg.com/vi/OuzUyrcvTg4/sddefault.jpg",
                    "width": 640,
                    "height": 480
                  },
                  "maxres": {
                    "url": "https://i.ytimg.com/vi/OuzUyrcvTg4/maxresdefault.jpg",
                    "width": 1280,
                    "height": 720
                  }
                },
                "channelTitle": "Pilot Debrief",
                "tags": [
                  "impossible turn",
                  "engine failure on takeoff",
                  "airplane stalls",
                  "plane loses engine on takeoff",
                  "plane stalls after takeoff",
                  "plane stalls and crashes",
                  "plane engine stalls",
                  "airplane crash",
                  "engine failure",
                  "airplane crash landing",
                  "airplane crash investigation",
                  "aviation",
                  "pilot",
                  "flying",
                  "Pilot debrief"
                ],
                "categoryId": "22",
                "liveBroadcastContent": "none",
                "defaultLanguage": "en",
                "localized": {
                  "title": "Pilot Tries Impossible Turn!",
                  "description": """The ibla babllabllablab """
                },
                "defaultAudioLanguage": "en"
              }
            }
          }
        }
      },

BUT -

now as another user - "johnpope" I want to follow user id "blabla" I use the helper.

tst_manager.follow(my_actor_id,following_actor_id)

and now I want to see the feeds for johnpope - (like facebook feeds / friends did posts / liked stuff etc)

this api is simply using the aggregator

    tst_date_weight_aggregator = DateWeightAggregator("cquiros") # is equal to the q str in uri 
    test = tst_manager.get_feeds(tst_date_weight_aggregator)
    print("test:",test)

do I need something else?

Screenshot 2023-08-23 at 10 22 31 pm

I dug into the other tests - and see this. https://github.com/qlands/elasticfeeds/blob/master/elasticfeeds/tests/test_02_aggregators.py#L37C3-L39C54

here - would substituting "cquiros" with "johnpope" basically be the equivalent of viewing the feeds for johnpope? That would contain

  # Test recent object type aggregator
    tst_date_weight_aggregator = DateWeightAggregator("cquiros")
    tst_manager.get_feeds(tst_date_weight_aggregator)

do I want to target the feeds or network here for a feed for johnpope (which would just show "blabla" user?

maybe with the DateWeightAggregator we could make show the variable name -

UPDATE

    tst_date_weight_aggregator = DateWeightAggregator(actor_id="cquiros")
    tst_manager.get_feeds(tst_date_weight_aggregator)

UPDATE for sanity test -


GET _search
{
  "size": 10,
  "query": {
    "match": {
        "_index": "testnetwork"
    }
  }
}

I can see the testnetwork

and I can see the uuid doesn't make sense - and I can easily delete it.


POST testnetwork/_delete_by_query
{
  "size": 10,
  "query": {
    "match": {
        "_id": "233b500c-432c-49e0-91d4-33290ea2b071"
    }
  }
}

tst_linked_activity = LinkedActivity(activity_id=str(uuid.uuid4())) // obviously wrong - must match the actor_id ??

this is the code


    tst_manager = Manager(
        feed_index = "testfeeds",
        network_index = "testnetwork",
        byo_connection = True
    )
    tst_manager.assign_connection(opensearch)
    tst_manager.follow("johndpope","blabla") # just use this helper instead of sample code which incorrectly uses cquiros 2x. should follow- edoquiros
Screenshot 2023-08-23 at 10 04 05 pm
johndpope commented 5 months ago

sorry for above guff - I close this ticket tomorrow on other account. I basically needed this simple facebook feed test case.

in this test:

Sally follows Bob but not Bill. Both Bob and Bill have activities (like liking a book or a movie). When fetching the feed for Sally, only Bob's activities should appear because Sally follows Bob but not Bill.


 # Define users Sally, Bob, and Bill
    sally = Actor("sally", "person")
    bob = Actor("bob", "person")
    bill = Actor("bill", "person")

    # Sally follows Bob
    tst_manager.follow("sally", "bob", now)

    # Bob likes a book
    book = Object("book1", "book")
    bob_likes_book = Activity("like", bob, book, published=now)
    tst_manager.add_activity_feed(bob_likes_book)

    # Bill likes a movie (Sally does not follow Bill)
    movie = Object("movie1", "movie")
    bill_likes_movie = Activity("like", bill, movie, published=now)
    tst_manager.add_activity_feed(bill_likes_movie)

    # Fetch and print activities for users Sally follows
    sally_feed_aggregator = RecentTypeAggregator("sally")
    sally_feed = tst_manager.get_feeds(sally_feed_aggregator)
    print("Sally's feed:", sally_feed)

Sally's feed: [{'type': 'like', 'activities': [{'actor': {'id': 'bob', 'type': 'person'}, 'published': '2024-04-04T23:19:06.416193', 'object': {'id': 'book1', 'type': 'book'}}]}]