Refactoring: Remote Functions

[cdharris]

Remote Functions Refactor

Overview

Remote functions exist to enable background->tab, and tab->background function delegation (i.e. Call a function in a tab, but have it run in the background and vice versa).

Refactoring has been started to make these functions interfaced and type safe.

Tasks

Grepping the codebase for remoteFunction( shows where it's still being used.

grep -R remoteFunction\(   
popup/sidebar-button/actions.ts:    await remoteFunction('showSidebar')()
popup/actions.ts:const fetchPageTagsRPC = remoteFunction('fetchPageTags')
popup/actions.ts:const fetchListsRPC = remoteFunction('fetchListPagesByUrl')
popup/actions.ts:const fetchAllListsRPC = remoteFunction('fetchAllLists')
popup/actions.ts:const fetchInitTagSuggRPC = remoteFunction('extendedSuggest')
popup/actions.ts:const isURLBlacklistedRPC = remoteFunction('isURLBlacklisted')
popup/actions.ts:const fetchInternalTabRPC = remoteFunction('fetchTab')
popup/actions.ts:const fetchTabByUrlRPC = remoteFunction('fetchTabByUrl')
popup/blacklist-button/actions.ts:const addToBlacklistRPC: (url: string) => Promise<void> = remoteFunction(
popup/blacklist-button/actions.ts:const processEventRPC: (args: any) => Promise<void> = remoteFunction(
popup/blacklist-button/actions.ts:const deletePagesRPC = remoteFunction('delPages')
popup/blacklist-button/actions.ts:const deletePagesByDomainRPC = remoteFunction('delPagesByDomain')
popup/blacklist-button/actions.ts:            () => remoteFunction('dispatchNotification')('db_error'),
popup/container.tsx:    processEvent = remoteFunction('processEvent')
popup/notif-button/actions.ts:const fetchUnreadCountRPC = remoteFunction('fetchUnreadCount')
popup/tooltip-button/actions.ts:const processEventRPC = remoteFunction('processEvent')
popup/bookmark-button/actions.ts:            () => remoteFunction('dispatchNotification')('db_error'),
popup/pause-button/actions.ts:const processEventRPC = remoteFunction('processEvent')
popup/pause-button/actions.ts:const togglePauseRPC = remoteFunction('toggleLoggingPause')
notifications/actions.ts:const fetchUnreadNotifications = remoteFunction('fetchUnreadNotifications')
notifications/actions.ts:const fetchReadNotifications = remoteFunction('fetchReadNotifications')
notifications/actions.ts:const fetchUnreadCount = remoteFunction('fetchUnreadCount')
notifications/actions.ts:const readNotification = remoteFunction('readNotification')
notifications/actions.ts:const processEvent = remoteFunction('processEvent')
notifications/container.tsx:const processEvent = remoteFunction('processEvent')
social-integration/actions.ts:const fetchSocialPostTags = remoteFunction('fetchSocialPostTags')
social-integration/actions.ts:const fetchInitTagSuggRPC = remoteFunction('extendedSuggest')
social-integration/actions.ts:const fetchSocialPostLists = remoteFunction('fetchSocialPostLists')
social-integration/actions.ts:const fetchAllListsRPC = remoteFunction('fetchAllLists')
social-integration/actions.ts:const addBookmarkRPC = remoteFunction('addSocialBookmark')
social-integration/actions.ts:const delBookmarkRPC = remoteFunction('delSocialBookmark')
social-integration/actions.ts:const addTweetRPC = remoteFunction('addTweet')
social-integration/twitter/components/save-to-memex-container.tsx:            const postTags = await remoteFunction('fetchSocialPostTags')({
social-integration/twitter/components/save-to-memex-container.tsx:                ? await remoteFunction('delSocialPages')([this.url])
annotations/annotations-manager.ts:        this._processEventRPC = remoteFunction('processEvent')
annotations/annotations-manager.ts:        this._createAnnotationRPC = remoteFunction('createAnnotation')
annotations/annotations-manager.ts:        this._addAnnotationTagRPC = remoteFunction('addAnnotationTag')
annotations/annotations-manager.ts:        this._getAllAnnotationsByUrlRPC = remoteFunction(
annotations/annotations-manager.ts:        this._getTagsByAnnotationUrlRPC = remoteFunction('getAnnotationTags')
annotations/annotations-manager.ts:        this._editAnnotationRPC = remoteFunction('editAnnotation')
annotations/annotations-manager.ts:        this._editAnnotationTagsRPC = remoteFunction('editAnnotationTags')
annotations/annotations-manager.ts:        this._deleteAnnotationRPC = remoteFunction('deleteAnnotation')
annotations/annotations-manager.ts:        this._bookmarkAnnotationRPC = remoteFunction('toggleAnnotBookmark')
annotations/annotations-manager.ts:        this._searchAnnotationsRPC = remoteFunction('searchAnnotations')
annotations/index.ts:        (await remoteFunction('getAllAnnotationsByUrl')({
common-ui/crowdfunding/components/CFModal.tsx:    private processEventRPC = remoteFunction('processEvent')
common-ui/crowdfunding/components/CFBox.tsx:    private _processEventRPC = remoteFunction('processEvent')
common-ui/crowdfunding/components/CFBox.tsx:    private _openLearnMoreUrl = remoteFunction('openLearnMoreTab')
common-ui/containers/IndexDropdown.tsx:        this.suggestRPC = remoteFunction('suggest')
common-ui/containers/IndexDropdown.tsx:        this.delTagRPC = remoteFunction(this.delTagRPCName)
common-ui/containers/IndexDropdown.tsx:        this.addTagsToOpenTabsRPC = remoteFunction('addTagsToOpenTabs')
common-ui/containers/IndexDropdown.tsx:        this.delTagsFromOpenTabsRPC = remoteFunction('delTagsFromOpenTabs')
common-ui/containers/IndexDropdown.tsx:        this.processEvent = remoteFunction('processEvent')
common-ui/containers/IndexDropdown.tsx:        this.fetchUserSuggestionsRPC = remoteFunction('fetchUserSuggestions')
common-ui/containers/IndexDropdown.tsx:        this.fetchHashtagSuggestionsRPC = remoteFunction(
common-ui/containers/IndexDropdown.tsx:                () => remoteFunction('dispatchNotification')('db_error'),
common-ui/containers/IndexDropdown.tsx:        return remoteFunction(rpcName)
common-ui/containers/MigrationNotice.js:        this.startMigration = remoteFunction('startMigration')
common-ui/containers/MigrationNotice.js:        this.isMigrating = remoteFunction('isMigrating')
common-ui/containers/AddListDropdownContainer.tsx:        this.addListRPC = remoteFunction('createCustomList')
common-ui/containers/AddListDropdownContainer.tsx:        this.addPageToListRPC = remoteFunction(props.addPageToListRPC)
common-ui/containers/AddListDropdownContainer.tsx:        this.deletePageFromListRPC = remoteFunction(props.delPageFromListRPC)
common-ui/containers/AddListDropdownContainer.tsx:        this.addOpenTabsToListRPC = remoteFunction('addOpenTabsToList')
common-ui/containers/AddListDropdownContainer.tsx:        this.removeOpenTabsFromListRPC = remoteFunction(
common-ui/containers/AddListDropdownContainer.tsx:        this.fetchListByIdRPC = remoteFunction('__fetchListById')
common-ui/containers/AddListDropdownContainer.tsx:        this.fetchListNameSuggestionsRPC = remoteFunction(
common-ui/containers/AddListDropdownContainer.tsx:                () => remoteFunction('dispatchNotification')('db_error'),
search-injection/content_script.js:const requestSearch = remoteFunction('search')
search-injection/components/container.tsx:        this.trackEvent = remoteFunction('trackEvent')
search-injection/components/container.tsx:        this.readNotification = remoteFunction('readNotification')
search-injection/components/container.tsx:        this.fetchNotifById = remoteFunction('fetchNotifById')
search-injection/components/container.tsx:        this.processEvent = remoteFunction('processEvent')
search-injection/components/container.tsx:        this.openOverviewRPC = remoteFunction('openOverviewTab')
search-injection/components/Dropdown.js:    openOptionsRPC = remoteFunction('openOptionsTab')
search-filters/actions.ts:    const tags = await remoteFunction('extendedSuggest')({
search-filters/actions.ts:    const suggestedHashtags = await remoteFunction('fetchAllHashtags')({
search-filters/actions.ts:    const suggestedDomains = await remoteFunction('extendedSuggest')({
search-filters/actions.ts:    const suggestedUsers: User[] = await remoteFunction('fetchAllUsers')({
analytics/store-tracking-option.js:        const processEvent = remoteFunction('processEvent')({
toolbar-notification/content_script/react/container.tsx:    openOptionsTab = remoteFunction('openOptionsTab')
toolbar-notification/content_script/react/container.tsx:    processEventRPC = remoteFunction('processEvent')
backup-restore/content_script/index.ts:        remoteFunction('storeWordpressUserId')(userId)
backup-restore/ui/backup-status-bar/components/StatusOverlay.tsx:        const hasInitialBackup = await remoteFunction('hasInitialBackup')()
backup-restore/ui/backup-status-bar/components/StatusOverlay.tsx:        const automaticBackupEnabled = await remoteFunction(
backup-restore/ui/backup-status-bar/BackupStatusBarContainer.tsx:            isAutomaticBackupAllowed: await remoteFunction(
backup-restore/ui/backup-status-bar/BackupStatusBarContainer.tsx:            isAutomaticBackupEnabled: await remoteFunction(
backup-restore/ui/backup-status-bar/BackupStatusBarContainer.tsx:            backupTimes: await remoteFunction('getBackupTimes')(),
backup-restore/ui/backup-status-bar/BackupStatusBarContainer.tsx:            backupLocation: await remoteFunction('getBackendLocation')(),
backup-restore/ui/backup-status-bar/BackupStatusBarContainer.tsx:            hasInitialBackup: await remoteFunction('hasInitialBackup')(),
backup-restore/ui/utils.js:    window.location.href = await remoteFunction('getBackupProviderLoginLink')({
backup-restore/ui/backup-pane/panes/running-process.tsx:        const info = await remoteFunction(this.props.functionNames.info)()
backup-restore/ui/backup-pane/panes/running-process.tsx:        await remoteFunction(this.props.functionNames.start)()
backup-restore/ui/backup-pane/panes/running-process.tsx:            const errorId = await remoteFunction(
backup-restore/ui/backup-pane/panes/running-process.tsx:        remoteFunction(this.props.functionNames.pause)()
backup-restore/ui/backup-pane/panes/running-process.tsx:        remoteFunction(this.props.functionNames.resume)()
backup-restore/ui/backup-pane/panes/running-process.tsx:        await remoteFunction(this.props.functionNames.cancel)()
backup-restore/ui/backup-pane/panes/setup-location.jsx:        const initialBackup = await remoteFunction('hasInitialBackup')()
backup-restore/ui/backup-pane/panes/setup-location.jsx:        const backendLocation = await remoteFunction('getBackendLocation')()
backup-restore/ui/backup-pane/panes/setup-location.jsx:                            await remoteFunction('forgetAllChanges')()
backup-restore/ui/backup-pane/panes/setup-location.jsx:                            await remoteFunction('forgetAllChanges')()
backup-restore/ui/backup-pane/panes/setup-size.jsx:                    estimation: await remoteFunction(
backup-restore/ui/backup-pane/panes/setup-size.jsx:            backendLocation: await remoteFunction('getBackendLocation')(),
backup-restore/ui/backup-pane/panes/overview.tsx:        const backupTimes = await remoteFunction('getBackupTimes')()
backup-restore/ui/backup-pane/panes/overview.tsx:        const hasInitialBackup = await remoteFunction('hasInitialBackup')()
backup-restore/ui/backup-pane/panes/overview.tsx:        const backupLocation = await remoteFunction('getBackendLocation')()
backup-restore/ui/backup-pane/panes/overview.tsx:        const automaticBackupEnabled = await remoteFunction(
backup-restore/ui/backup-pane/container.logic.ts:    const isAuthenticated = await remoteFunction(
backup-restore/ui/backup-pane/container.logic.ts:    const backendLocation = await remoteFunction('getBackendLocation')()
backup-restore/ui/backup-pane/container.logic.ts:    const hasInitialBackup = !!(await remoteFunction('hasInitialBackup')())
backup-restore/ui/backup-pane/container.logic.ts:        remoteFunction('setBackupBlobs')(event.saveBlobs)
backup-restore/ui/backup-pane/container.logic.ts:                    remoteFunction('hasInitialBackup')(),
backup-restore/ui/backup-pane/container.logic.ts:                    remoteFunction('getBackupInfo')(),
backup-restore/ui/backup-pane/container.logic.ts:                    remoteFunction('getBackendLocation')(),
backup-restore/ui/backup-pane/container.logic.ts:                remoteFunction('setBackendLocation')(location)
backup-restore/ui/backup-pane/container.logic.ts:                const isAutomaticBackupEnabled = await remoteFunction(
backup-restore/ui/backup-pane/container.logic.ts:                    await remoteFunction('enableAutomaticBackup')
backup-restore/ui/backup-pane/container.logic.ts:                await remoteFunction('initRestoreProcedure')(location)
direct-linking/content_script/index.js:        remoteFunction('followAnnotationRequest')()
direct-linking/content_script/interactions.ts:    const result: { url: string } = await remoteFunction('createDirectLink')({
direct-linking/background/index.ts:        await remoteFunction(functionName, { tabId: currentTab.id })(...args)
overview/results/actions.ts:const processEventRPC = remoteFunction('processEvent')
overview/results/actions.ts:const createSocialBookmarkRPC = remoteFunction('addSocialBookmark')
overview/results/actions.ts:const deleteSocialBookmarkRPC = remoteFunction('delSocialBookmark')
overview/results/actions.ts:            () => remoteFunction('dispatchNotification')('db_error'),
overview/tooltips/actions.ts:const processEventRPC = remoteFunction('processEvent')
overview/search-bar/actions.ts:const processEventRPC = remoteFunction('processEvent')
overview/search-bar/actions.ts:const pageSearchRPC = remoteFunction('searchPages')
overview/search-bar/actions.ts:const annotSearchRPC = remoteFunction('searchAnnotations')
overview/search-bar/actions.ts:const socialSearchRPC = remoteFunction('searchSocial')
overview/search-bar/components/HeaderContainer.tsx:const processEventRPC = remoteFunction('processEvent')
overview/search-bar/components/DateRangeSelection.tsx:const processEvent = remoteFunction('processEvent')
overview/delete-confirm-modal/actions.ts:const processEventRPC = remoteFunction('processEvent')
overview/delete-confirm-modal/actions.ts:const deletePagesRPC = remoteFunction('delPages')
overview/delete-confirm-modal/actions.ts:const deleteSocialPagesRPC = remoteFunction('delSocialPages')
overview/delete-confirm-modal/actions.ts:            () => remoteFunction('dispatchNotification')('db_error'),
overview/onboarding/popup-helper.ts:const processEventRPC = remoteFunction('processEvent')
overview/onboarding/popup-helper.ts:const openOptionsRPC = remoteFunction('openOptionsTab')
util/webextensionRPC.test.ts:        const remoteFunc = remoteFunction('remoteFunc', { tabId: 1 })
util/webextensionRPC.test.ts:    //     const remoteFunc = remoteFunction('remoteFunc', { tabId: 1 })
util/webextensionRPC.test.ts:        const remoteFunc = remoteFunction('remoteFunc', { tabId: 1 })
util/webextensionRPC.test.ts:        const remoteFunc = remoteFunction('remoteFunc')
util/webextensionRPC.test.ts:    //     const remoteFunc = remoteFunction('remoteFunc', { tabId: 1 })
util/webextensionRPC.test.ts:    //     const remoteFunc = remoteFunction('remoteFunc', { tabId: 1 })
util/webextensionRPC.test.ts:        const remoteFunc = remoteFunction('remoteFunc', { tabId: 1 })
util/webextensionRPC.ts:// const myRemoteFunc = remoteFunction('myFunc')
util/webextensionRPC.ts:                return _remoteFunction(property.toString())(...args)
util/webextensionRPC.ts:                return _remoteFunction(property.toString(), { tabId })(...args)
util/webextensionRPC.ts:export function remoteFunction(
util/webextensionRPC.ts:    return _remoteFunction(funcName, { tabId })
util/webextensionRPC.ts:function _remoteFunction(funcName: string, { tabId }: { tabId?: number } = {}) {
options/imports/actions.ts:const processEvent = remoteFunction('processEvent')
options/tutorial/index.tsx:    private processEventRPC = remoteFunction('processEvent')
options/blacklist/actions.ts:const deletePagesByPattern = remoteFunction('delPagesByPattern')
options/blacklist/actions.ts:const getMatchingPageCount = remoteFunction('getMatchingPageCount')
options/blacklist/actions.ts:const dirtyEstsCache = remoteFunction('dirtyEstsCache')
options/blacklist/actions.ts:const processEvent = remoteFunction('processEvent')
options/blacklist/actions.ts:            () => remoteFunction('dispatchNotification')('db_error'),
activity-logger/background/tab-state.ts:        return remoteFunction('toggleIFrameRender', {
in-page-ui/tooltip/content_script/interactions.ts:const openOptionsRPC = remoteFunction('openOptionsTab')
in-page-ui/tooltip/onboarding-interactions.ts:const processEventRPC = remoteFunction('processEvent')
in-page-ui/sidebar/react/components/congrats-message.tsx:    openOptionsTab = remoteFunction('openOptionsTab')
sidebar-overlay/ribbon/components/ribbon.tsx:        this.openOverviewTabRPC = remoteFunction('openOverviewTab')
sidebar-overlay/ribbon/components/ribbon.tsx:        this.openOptionsTabRPC = remoteFunction('openOptionsTab')
sidebar-overlay/sidebar/actions.ts:const processEventRPC = remoteFunction('processEvent')
sidebar-overlay/sidebar/actions.ts:            () => remoteFunction('dispatchNotification')('db_error'),
sidebar-overlay/annotation-box/annotation-box-container.tsx:    private _processEventRPC = remoteFunction('processEvent')
sidebar-overlay/components/congrats-message.tsx:    openOptionsTab = remoteFunction('openOptionsTab')
sidebar-overlay/utils.ts:const openOptionsTabRPC = remoteFunction('openOptionsTab')
sidebar-overlay/utils.ts:export const toggleSidebarOverlay = remoteFunction('toggleSidebarOverlay')
custom-lists/actions.js:        await remoteFunction(delPageFromListRPC)({
custom-lists/actions.js:        const lists = await remoteFunction('fetchAllLists')({
custom-lists/actions.js:            await remoteFunction('fetchListIgnoreCase')({ name }),
custom-lists/actions.js:            const id = await remoteFunction('createCustomList')({ name })
custom-lists/actions.js:        await remoteFunction('updateListName')({ id, newName, oldName })
custom-lists/actions.js:        await remoteFunction('removeList')({ id })
custom-lists/actions.js:        await remoteFunction(addPagetoListRPC)({ id, url })

Prior understanding and existing system

Following the format of browser extensions, Memex exists as scripts that are run in pages on tabs (content_script.js), and the parts that run in an 'invisible tab' in the background outside of any page (background.js).

To communicate between code on a page and code outside the page, a custom RPC wrapper is setup (src/util/webextensionRPC.ts) around the native WebExt API which enables extensions to communicate across scripts.

Currently, code is structured so that functions that are intended to be run in the background script, are passed into the function makeRemotelyCallable, which registers the given function name in a variable remotelyCallableFunctions, responsible for looking these functions up.

This variable referencing functions that can be run in the background is referenced by incomingRPCListener setup to run on the browser.runtime (the background extension script) with browser.runtime.onMessage.addListener(incomingRPCListener)

Background script functionality commonly resides in /src/{feature}/background/index.js and they are registered from src/background.js

An example of how this works is as follows:

The extension starts it's background page, registering these background functions, one of which is done by calling makeRemotelyCallable({funcA}), where funcA is a function imported from somewhere, e.g. const funcA = (arg1,arg2) => { return 'done'}.

At some point from the content script run in a page, that funcA is called but is run not in the content script that calls it, but in the background script, by: const result = await remoteFunction('funcA')('1','2').

Further functionality: These remoteFunction calls also support running functions the other way around too. If called instead with const result = await remoteFunction('FuncA',{tabId: 2})() the function funcA will actually be run in the script on the page indicated by tabId. This is used for example, when needing to extract content on the page, by functionality that is running in the background script.

New system

To assist the incremental refactoring, the old functions makeRemotelyCallable and remoteFunction still exist but should be considered depreciated. Their new typesafe versions are makeRemotelyCallableType and runInBackground. (we can rename makeRemotelyCallableType back to makeRemotelyCallable if desired once all refactoring has been done).

The runInBackground<T> function returns a typed proxy object. Requests for a property (including a method) on that proxy, actually call a remote function, so calls to

const examples = runInBackground<ExampleInterface>({example1: () => false,example2:() => false);
const result = await example.example1()

will actually call remoteFunction('example1')() under the hood.

Work description

Refactor these remote functions to be type safe. Including the setup of these functions (makeRemotelyCallable) and the client usage of these functions (remoteFunction).

Methodology

• Identify a set of remote function registrations from the TODO list below.

• Create an interface that describes the functions, arguments and returns, or a set thereof, e.g. NotificationInterface. Interfaces should be defined in a standalone types file or inside an existing standalone file with only types, so that when importing this interface, it doesn't import the functionality too, and remains lightweight.

• Modify where it is setup via makeRemotelyCallable to use makeRemotelyCallableType<T> where T is this newly created interface.

• Move the registration of this remote function to alongside the others in src/background.js

• Search through all usages of this function from remoteFunction. (N.B. usage may not be directly done using a string literal e.g. remoteFunction('exampleFunc'), it may be using a variable, e.g. const func = 'exampleFunc'; remoteFunction(func)).

  -   Change usages to `runInBackground<T>` where `T` is this newly created interface.
  
  -   Change the usages to not each call `runInBackground` to create the function directly, but rather import from `src/util/remote-functions-background.ts`. Using the created proxy object (returned from `runInBackground`) transparently as if it were a concrete implementation. e.g. instead of calling `runInBackground('addBookmark')(args)` from within some feature's functionality, it uses `bookmarks.addBookmark(args)` where `bookmarks` is assigned to the created proxy object in `remote-functions-background.ts`, in the same manner the others are.

• Test

• Update this documentation with progress and any inconsistencies found.

• Commit regularly

[ShishKabab]

One thing that is still missing in the new RPC system is the inclusion of functions that need that tab argument in makeRemotelyCallableType.