Firebase Cloud functions is slow

[alexanderkhitev]

Hi firebase team! We use Firebase as a backend for our service. It has a PeopleViewController that displays users within a radius of the current user. The radius can vary from 1 to 40 miles. Also there is a check for blocked users, age, sex. At the very beginning of the development of the project, a temporary implementation was made for this capability within the application. I created several managers who fulfilled these requests, and then gave the result to PeopleViewController. At the beginning of the project, because of the tight deadlines, pagination was not implemented, but the pagination was not particularly needed since the users even in the radius were less than 100. Now, in one radius there can be more than 600 users and I think it is not right that these requests are performed inside application. We move these functions from the app Side to the cloud function so that we can adjust how many users should be in each issue. The problem is that even a warmed-up cloud function works slower than all functions related to this capability inside the application. On average, the code inside the application for a radius of 40 miles and the inclusion of all age groups returns 646 users in 1.5 seconds, while the best result is cloud functions 3.6. Below I will give examples of the time and code with the app side and cloud funciton.

The results are obtained using the code that is executed in the application:

1 number of users 646, 1.6775569915771484 seconds
2 number of users 646, 1.4022901058197021 seconds
3 number of users 646, 1.5957129001617432 seconds

The results with the cloud function time are specified in milliseconds. The parameters and location are the same as the code that is run from the application:

1 "timers": {
    "afterVerify": 1,
    "gotIdsFromGeofire": 1372,
    "gotBlockedUsersIDs": 2463,
    "gotBlockedByUsersIDs": 2467,
    "gotUserSearchLocationModels": 5256,
    "gotAllInitData": 5256,
    "filtersAndSortsDone": 6168
},
2 "timers": {
    "afterVerify": 6,
    "gotIdsFromGeofire": 1209,
    "gotBlockedUsersIDs": 1909,
    "gotBlockedByUsersIDs": 1913,
    "gotUserSearchLocationModels": 3800,
    "gotAllInitData": 3800,
    "filtersAndSortsDone": 4207
},
3 "timers": {
    "afterVerify": 1,
    "gotIdsFromGeofire": 812,
    "gotBlockedUsersIDs": 1415,
    "gotBlockedByUsersIDs": 1419,
    "gotUserSearchLocationModels": 3912,
    "gotAllInitData": 3913,
    "filtersAndSortsDone": 4417
},

In the managers who request a list of users and filter the received data within the application.

extension PeopleListViewController {

    @objc fileprivate func requestPeopleNearBy() {
        let firPeopleSearchDatabaseManagerStart = Date().timeIntervalSince1970
        firPeopleSearchDatabaseManager.searchPeople(success: {  [weak self](userSearchLocationModels) in
            let firPeopleSearchDatabaseManagerEnd = Date().timeIntervalSince1970
            let resultTimeStamp = firPeopleSearchDatabaseManagerEnd - firPeopleSearchDatabaseManagerStart
            debugPrint("firPeopleSearchDatabaseManager userSearchLocationModels", userSearchLocationModels.count, "resultTimeStamp", resultTimeStamp)
        }) { (error) in

        }
    }

}

class FIRPeopleSearchDatabaseManager {

    fileprivate enum MainGateways {
        case userSearchLocationModel, userLocations

        var description: String {
            switch self {
            case .userSearchLocationModel:
                return "userSearchLocationModel"
            case .userLocations:
                return "userLocations"
            }
        }
    }

    func saveUserSearchLocationModel(_ userSearchLocationModel: UserSearchLocationModel, success: (() -> Void)?, fail: ((_ error: Error) -> Void)?) {
        DispatchQueue.global(qos: .background).async {
            let ref = Database.database().reference().child(MainGateways.userSearchLocationModel.description).child(userSearchLocationModel.userID)
            let json = userSearchLocationModel.toJSON()

            ref.updateChildValues(json, withCompletionBlock: { (error, ref) in
                guard error == nil else {
                    fail?(error!)
                    return
                }
                success?()
            })
        }
    }

    private func downloadUserSearchLocationModel(_ userID: String, success: ((_ userSearchLocationModel: UserSearchLocationModel) -> Void)?, notExist: (() -> Void)?, fail: ((_ error: Error) -> Void)?) {
        DispatchQueue.global(qos: .background).async {
            let ref = Database.database().reference().child(MainGateways.userSearchLocationModel.description).child(userID)
            ref.observeSingleEvent(of: .value, with: { (snapshot) in
                if snapshot.value is NSNull {
                    notExist?()
                    return
                }
                guard let json = snapshot.value as? [String : Any] else {
                    debugPrint("doest exist", userID)
                    notExist?()
                    return
                }
                guard let userSearchLocationModel = Mapper<UserSearchLocationModel>().map(JSON: json) else {
                    debugPrint("doest exist", userID)
                    notExist?()
                    return
                }
                guard !userSearchLocationModel.userID.isEmpty else {
                    notExist?()
                    return
                }
                success?(userSearchLocationModel)
            }, withCancel: { (error) in
                fail?(error)
            })
        }
    }

    func searchPeople(success: ((_ searchLocationModels: [UserSearchLocationModel]) -> Void)?, fail: ((_ error: Error) -> Void)?) {
        let realmManager = RealmManager()
        guard let currentUserID = realmManager.getCurrentUser()?.id else { return }
        guard let location = LocationManager.shared.currentLocation else { return }
        let realmUserSettingsManager = RealmUserSettingsManager()
        let miles = realmUserSettingsManager.getMaxDistanceInMiles() ?? 15

        let distanceConverter = DistanceConvertor()
        let radius = distanceConverter.convertedMilesToMetersForFIRRequest(miles)

        DispatchQueue.global(qos: .background).async {
            let usersGeofireRef = Database.database().reference().child(MainGateways.userLocations.description)
            guard let geoFire = GeoFire(firebaseRef: usersGeofireRef) else { return }
            guard let circleQuery = geoFire.query(at: location, withRadius: radius) else { return }

            var usersListIDs = [String]()
            var generalBlockModel = [BlockModel]()
            let dispatchGroup = DispatchGroup()

            let blockSystemManager = BlockSystemManager()
            dispatchGroup.enter()
            blockSystemManager.requestBlockingUserIDs(completion: { (blockModels, error) in
                generalBlockModel = blockModels
                dispatchGroup.leave()
            })

            dispatchGroup.enter()
            circleQuery.observe(.keyEntered, with: { (key, location) in
                if let _key = key {
                    if _key != currentUserID {
                        usersListIDs.append(_key)
                    }
                }
            })

            circleQuery.observeReady({
                circleQuery.removeAllObservers()
                dispatchGroup.leave()
            })

            dispatchGroup.notify(queue: .global(qos: .background), execute: {
                if usersListIDs.count == 0 {
                    circleQuery.removeAllObservers()
                    success?([])
                    return
                }

                var unblockedUserListIDs = [String]()
                for usersListID in usersListIDs {
                    let isContains = generalBlockModel.contains(where: { $0.userID == usersListID })
                    if !isContains {
                        unblockedUserListIDs.append(usersListID)
                    }
                }
                self.downloadUsers(usersListIDs: unblockedUserListIDs, success: success)
            })
        }
    }

    private func downloadUsers(usersListIDs: [String], success: ((_ searchLocationModels: [UserSearchLocationModel]) -> Void)?) {
        var userIDsCount = usersListIDs.count
        var userSearchLocationModels = [UserSearchLocationModel]()

        for userID in usersListIDs {
            self.downloadUserSearchLocationModel(userID, success: { (userSearchLocationModel) in
                userSearchLocationModels.append(userSearchLocationModel)
                if userIDsCount == userSearchLocationModels.count {
                    self.filter(userSearchLocationModels, success: { (sortedUserSearchLocationModels) in
                        success?(sortedUserSearchLocationModels)
                    })
                }
            }, notExist: {
                userIDsCount -= 1
                if userIDsCount == userSearchLocationModels.count {
                    self.filter(userSearchLocationModels, success: { (sortedUserSearchLocationModels) in
                        success?(sortedUserSearchLocationModels)
                    })
                }
            }, fail: { (error) in
                userIDsCount -= 1
                if userIDsCount == userSearchLocationModels.count {
                    self.filter(userSearchLocationModels, success: { (sortedUserSearchLocationModels) in
                        success?(sortedUserSearchLocationModels)
                    })
                }
            })
        }
    }

    private func filter(_ userSearchLocationModels: [UserSearchLocationModel], success: ((_ sortedUserSearchLocationModels: [UserSearchLocationModel]) -> Void)?) {
        filterByUserSearchFilter(userSearchLocationModels) { (searchFilteredUserSearchLocationModels) in

            self.filterByOnlineAndRecentLaunchApp(searchFilteredUserSearchLocationModels, success: { (onlineRecentFilteredUserSearchLocationModels) in
                success?(onlineRecentFilteredUserSearchLocationModels)
            })
        }
    }

    private func filterByUserSearchFilter(_ userSearchLocationModels: [UserSearchLocationModel], success: ((_ sortedUserSearchLocationModels: [UserSearchLocationModel]) -> Void)?) {
        DispatchQueue.main.async {
            let timeManager = TimeManager()
            let realmUserSettingsManager = RealmUserSettingsManager()
            guard let cuPeopleFilterSettings = realmUserSettingsManager.getUserPeopleFilterSettings() else { return }
            var sortedUserSearchLocationModels = [UserSearchLocationModel]()

            for userSearchLocationModel in userSearchLocationModels {
                let userYearAge = timeManager.getUserAgeFromBirthdayTimeStamp(userSearchLocationModel.birthdayTimeStamp)
                if cuPeopleFilterSettings.filterGenderModeIndex != 2 {
                    // sorting with mode
                    if cuPeopleFilterSettings.maxAgeValue != UserPeopleFilterSettings.Standard.maxAge {
                        if userYearAge >= cuPeopleFilterSettings.minAgeValue  && userYearAge <= cuPeopleFilterSettings.maxAgeValue && userSearchLocationModel.genderIndex == cuPeopleFilterSettings.filterGenderModeIndex  {
                            sortedUserSearchLocationModels.append(userSearchLocationModel)
                        }
                    } else {
                        // max age settings
                        if userYearAge >= cuPeopleFilterSettings.minAgeValue && userSearchLocationModel.genderIndex == cuPeopleFilterSettings.filterGenderModeIndex  {
                            sortedUserSearchLocationModels.append(userSearchLocationModel)
                        }
                    }
                } else {
                    if cuPeopleFilterSettings.maxAgeValue != UserPeopleFilterSettings.Standard.maxAge {
                        if userYearAge >= cuPeopleFilterSettings.minAgeValue  && userYearAge <= cuPeopleFilterSettings.maxAgeValue {
                            sortedUserSearchLocationModels.append(userSearchLocationModel)
                        }
                    } else {
                        // max age in settings
                        if userYearAge >= cuPeopleFilterSettings.minAgeValue {
                            sortedUserSearchLocationModels.append(userSearchLocationModel)
                        }
                    }
                }
            }

            //sorting by online status and last seen timestamp

            let onlineUsers = sortedUserSearchLocationModels.filter { $0.isOnline }
            let otherUsersSortedByTimeStamp = sortedUserSearchLocationModels.filter { !$0.isOnline }.sorted { $0.lastSeenTimeStamp > $1.lastSeenTimeStamp }

            var resultSortedArray = [UserSearchLocationModel]()

            resultSortedArray.append(contentsOf: onlineUsers)
            resultSortedArray.append(contentsOf: otherUsersSortedByTimeStamp)

            success?(resultSortedArray)
        }
    }

    /// 1. online users sorted by most recently active on the app 2. offline users sorted by most recently active on the app
    private func filterByOnlineAndRecentLaunchApp(_ userSearchLocationModels: [UserSearchLocationModel], success: ((_ sortedUserSearchLocationModels: [UserSearchLocationModel]) -> Void)?) {
        DispatchQueue.global(qos: .background).async {
            let sortedUsers = userSearchLocationModels.sorted(by: { $0.recentActivityTimeStamp > $1.recentActivityTimeStamp })

            success?(sortedUsers)
        }
    }

}

From BlockSystemManager

extension BlockSystemManager {

    func requestBlockingUserIDs(completion: ((_ blockModels: [BlockModel], _ error: Error?) -> Void)?) {
        DispatchQueue.main.async {
            guard let currentUserID = RealmManager().getCurrentUser()?.id else { return }
            DispatchQueue.global(qos: .background).async {
                let dispatchGroup = DispatchGroup()
                var generalBlockModels = [BlockModel]()

                let blockedUsersRef = Database.database().reference().child(MainPath.blockingInfo.description).child(currentUserID).child(SubPath.blockedUsers.description)

                dispatchGroup.enter()
                blockedUsersRef.observeSingleEvent(of: .value, with: { (snap) in
                    if snap.value is NSNull {
                        dispatchGroup.leave()
                        return
                    }
                    guard let dict = snap.value as? [String : [String : Any]] else {
                        dispatchGroup.leave()
                        return
                    }
                    guard let blockModelsDict = Mapper<BlockModel>().mapDictionary(JSON: dict)?.values else {
                        dispatchGroup.leave()
                        return
                    }
                    let blockModels = Array(blockModelsDict)
                    generalBlockModels += blockModels
                    dispatchGroup.leave()
                }, withCancel: { (error) in
                    dispatchGroup.leave()
                })

                let blockedByUsersRef = Database.database().reference().child(MainPath.blockingInfo.description).child(currentUserID).child(SubPath.blockedByUsers.description)
                dispatchGroup.enter()
                blockedByUsersRef.observeSingleEvent(of: .value, with: { (snap) in
                    if snap.value is NSNull {
                        dispatchGroup.leave()
                        return
                    }
                    guard let dict = snap.value as? [String : [String : Any]] else {
                        dispatchGroup.leave()
                        return
                    }
                    guard let blockModelsDict = Mapper<BlockModel>().mapDictionary(JSON: dict)?.values else {
                        dispatchGroup.leave()
                        return
                    }
                    let blockModels = Array(blockModelsDict)
                    generalBlockModels += blockModels
                    dispatchGroup.leave()
                }, withCancel: { (error) in
                    dispatchGroup.leave()
                })

                dispatchGroup.notify(queue: .global(qos: .background), execute: {
                    completion?(generalBlockModels, nil)
                })
            }
        }
    }

}

This function is located in the PeopleSearchManager and calls the cloud function

func searchPeopleFirstRequest(success: ((_ searchLocationModels: [UserSearchLocationModel]) -> Void)?, fail: ((_ error: Error) -> Void)?) {
    DispatchQueue.main.async {
        guard let location = LocationManager.shared.currentLocation else { return }
        let realmUserSettingsManager = RealmUserSettingsManager()
        let miles = realmUserSettingsManager.getMaxDistanceInMiles() ?? 15

        let parameters = ["miles" : miles, "latitude" : location.coordinate.latitude, "longitude" : location.coordinate.longitude] as [String : Any]
        DispatchQueue.global(qos: .background).async {
            Auth.auth().currentUser?.getIDTokenForcingRefresh(true, completion: { (token, error) in
                if let error = error {
                    // Handle error
                    fail?(error)
                    return
                }
                guard let _token = token else { return }
                let headers = ["X-Auth-MyApp-Token" : _token]

                guard let url = URL(string: BackendEndPoint.searchPeopleFirstRequest.path) else { return }
                AlamofireMyAppManager.shared.request(url, method: .get, parameters: parameters, encoding: URLEncoding.default, headers: headers).responseJSON(completionHandler: { (response) in
                    switch response.result {
                    case .success(let value):
                        guard let dict = value as? [String : Any] else {
                            success?([])
                            return
                        }
                        guard let result = dict["result"] as? [[String : Any]] else  {
                            success?([])
                            return
                        }
                        let userSearchLocationModels = Mapper<UserSearchLocationModel>().mapArray(JSONArray: result)
                        success?(userSearchLocationModels)
                        // for second request
                        if let lastUserSearchLocationModelID = dict["lastUserSearchLocationModelID"] as? String {
                            debugPrint("lastUserSearchLocationModelID", lastUserSearchLocationModelID)
                            self.peopleSecondRequestModel = PeopleSecondRequestModel(lastUserSearchLocationModelID: lastUserSearchLocationModelID, location: location, radius: miles, offset: userSearchLocationModels.count)
                        }
                    case .failure(let error):
                        debugPrint(error)
                        fail?(error)
                    }
                })
            })
        }
    }
}

Cloud function

const functions = require('firebase-functions');
const admin = require('firebase-admin');
const GeoFire = require('geofire');
const moment = require('moment');

const map = require('lodash').map;
const filter = require('lodash').filter;
const sortBy = require('lodash').sortBy;
const last = require('lodash').last;

module.exports = functions.https.onRequest((req, res) => {
    const token = req.header('X-Auth-MyApp-Token');
    const miles = req.query['miles'];
    const latitude = req.query['latitude'];
    const longitude = req.query['longitude'];

    let currentUserID;
    let outputUsers;

    const startTime = moment();
    const timers = {};

    admin.auth().verifyIdToken(token)

        .then((decodedToken) => {
            timers.afterVerify = moment() - startTime;
            currentUserID = decodedToken.uid;
            console.log('userID', currentUserID);

            const getUsersSearchModelsByLocationPromise = new Promise((resolve, reject) => {
                try {
                const userLocationRef = admin.database().ref().child('userLocations');
                const geoFire = new GeoFire(userLocationRef);

                const geoQuery = geoFire.query({
                center: [Number(latitude), Number(longitude)],
                radius: Number(miles) * 1.609
                });

                const usersIDs = [];

                const keyListener = geoQuery.on('key_entered', (key) => usersIDs.push(key));
                geoQuery.on('ready', () => {
                keyListener.cancel();
                geoQuery.cancel();
                timers.gotIdsFromGeofire = moment() - startTime;
                resolve(usersIDs)
                });

                } catch (error) {
                reject(error)
                }
                })
                .then((userIDs) => {
                    return Promise.all(map(userIDs, (id) => admin.database().ref().child('userSearchLocationModel').child(id)
                        .once('value')
                        .then((snapshot) => snapshot.val())
                    ))
                        .then((users) => {
                            timers.gotUserSearchLocationModels = moment() - startTime;
                            return Promise.resolve(users);
                            })
                    });

            const getBlockedUsersIDsPromise = Promise.all([
                admin.database().ref().child('blockingInfo').child(currentUserID).child('blockedUsers')
                    .once('value')
                    .then((snapshot) => snapshot.val())
                    .then((users) => {
                        timers.gotBlockedUsersIDs = moment() - startTime;
                        return map(users, 'userID')
                        }),

                admin.database().ref().child('blockingInfo').child(currentUserID).child('blockedByUsers')
                    .once('value')
                    .then((snapshot) => snapshot.val())
                    .then((users) => {
                        timers.gotBlockedByUsersIDs = moment() - startTime;
                        return map(users, 'userID')
                        }),

                ]);

            const getFilterSettingsPromise = admin.database().ref().child('userPeopleFilterSettings').child(currentUserID)
                .once('value')
                .then((snapshot) => snapshot.val());

            return Promise.all([
                getUsersSearchModelsByLocationPromise,
                getBlockedUsersIDsPromise,
                getFilterSettingsPromise
                ]);

            })
        .then(([users, blockedUsersIds, filterSettings]) => {
            timers.gotAllInitData = moment() - startTime;
            outputUsers = users;

            // filter nulls
            outputUsers = filter(outputUsers, (user) => user);

            // filter blocked
            outputUsers = filter(outputUsers, (user) => blockedUsersIds.indexOf(user.userID) === -1);

            // filter by gender
            outputUsers = filterSettings.filterGenderModeIndex !== 2 ?
                filter(outputUsers, (user) => user.genderIndex === filterSettings.filterGenderModeIndex) :
            outputUsers;

            // filter by age
            outputUsers = filter(outputUsers, (user) => {
                const userAge = moment().diff(user.birthdayTimeStamp  * 1000, 'years');
                if (filterSettings.maxAgeValue === 55 && userAge >= 55) return true;
                return (userAge >= filterSettings.minAgeValue && userAge <= filterSettings.maxAgeValue);
                });

            // sort by recent activity timestamp
            outputUsers = sortBy(outputUsers, 'recentActivityTimeStamp');

            // slice
            outputUsers = outputUsers.slice(0, 99);

            timers.filtersAndSortsDone = moment() - startTime;

            return res.status(200).send({
                timers: timers,
                result: outputUsers,
                lastUserSearchLocationModelID: last(outputUsers)
            });

            })
        .catch((error) => {
            console.log('Error: ', error);
            return res.status(500).send({
                code: error.code,
                message: error.message,
                stack: error.stack
            });
            });
    });

Also I want to say that I read this post, but as I wrote at the beginning, that even the heated function works like this. What is the reason for such a big difference in the result? How can this be remedied or what are the solutions to this problem?

[puf]

Please indicate when you cross-post: https://stackoverflow.com/questions/46215232/firebase-cloud-functions-is-slow.

[laurenzlong]

Closing since it is not following the issues template.