A skeleton class is a class developed in Ruby that inherits from Ruby Sequel and has direct access to the database.
Instances of skeleton classes work at the server side.
A stub class is a class with API communication with the server.
Instances of stub classes work at the client side.
Instances of stub classes have a counter-part at the server side who generate the API responses.
Any stub class inherits directly or indirectly from BlackStack::Base:
module BlackStack
# Base class.
# List of methods you have to overload if you develop a profile type.
#
class Base
# object json descriptor
attr_accessor :desc
def self.object_name
raise 'You have to overload this method in your class.'
end
def initialize(h)
self.desc = h
end
# Crate an instance of a child class using speicfications in the `desc['name']` attribute.
# By default, returns the same instance.
def child_class_instance
return self
end
# sysowner must provide the id_account for getting an account value.
# for non sysowner it is assigned to his account.
def self.account_value(id_account:nil, field:)
params = {}
params['id_account'] = id_account
params['field'] = field
# call the API
ret = BlackStack::API.post(
endpoint: "account_value",
params: params
)
raise "Error calling account_value endpoint: #{ret['status']}" if ret['status'] != 'success'
return ret['result']
end # def self.base
# Get array of hash descriptor of profile.
#
# Parameters:
# - id_account: guid. Id of the account to bring the profiles. Sysowner must provide the id_account for getting an account value. For non sysowner it is assigned to his account.
# - promiscuous: boolean. It works only for Sysowner. If true, it will bring all non-deleted rows, including the ones that are not owned by Sysowner. If false, it will bring only rows matching id_profile. Default: false.
# - page: integer. Page number.
# - limit: integer. Number of profiles per page.
# - params: hash. Additional filter parameters used by the specific child class.
#
def self.page(id_account: nil, promiscuous: false, page:, limit:, filters: {})
# add page and limit to the params
params = {}
params['id_account'] = id_account
params['promiscuous'] = promiscuous
params['page'] = page
params['limit'] = limit
params['filters'] = filters
params['backtrace'] = BlackStack::API.backtrace
# call the API
ret = BlackStack::API.post(
endpoint: "#{self.object_name}/page",
params: params
)
raise "Error calling page endpoint: #{ret['status']}" if ret['status'] != 'success'
return ret['results'].map { |h| self.new(h).child_class_instance }
end # def self.base
# Get array of hash descriptors of profiles.
#
# Parameters:
# - id: guid. Id of the profile to bring.
#
def self.count
params = {}
params['backtrace'] = BlackStack::API.backtrace
ret = BlackStack::API.post(
endpoint: "#{self.object_name}/count",
params: params
)
raise "Error calling count endpoint: #{ret['status']}" if ret['status'] != 'success'
return ret['result'].to_i
end # def self.count
# Get array of hash descriptors of profiles.
#
# Parameters:
# - id: guid. Id of the profile to bring.
#
def self.get(id)
params = {}
params['id'] = id
params['backtrace'] = BlackStack::API.backtrace
ret = BlackStack::API.post(
endpoint: "#{self.object_name}/get",
params: params
)
raise "Error calling get endpoint: #{ret['status']}" if ret['status'] != 'success'
return self.new(ret['result']).child_class_instance
end # def self.get
# Submit a hash descriptor to the server for an update
#
def self.update(desc)
params = {}
params['desc'] = desc
params['backtrace'] = BlackStack::API.backtrace
ret = BlackStack::API.post(
endpoint: "#{self.object_name}/update",
params: params
)
raise "Error calling update endpoint: #{ret['status']}" if ret['status'] != 'success'
return self.new(ret['result']).child_class_instance
end # def self.update
# Submit a hash descriptor to the server for an update of status only
#
def self.update_status(desc)
params = {}
params['desc'] = desc
params['backtrace'] = BlackStack::API.backtrace
ret = BlackStack::API.post(
endpoint: "#{self.object_name}/update_status",
params: params
)
raise "Error calling update_status endpoint: #{ret['status']}" if ret['status'] != 'success'
return
end # def self.update
# Submit a hash descriptor to the server for an update
#
def update
self.class.update(self.desc)
end
# Submit a hash descriptor to the server for an update of status only
#
def update_status
self.class.update(self.desc)
end
# Submit a hash descriptor to the server for an insert
#
def self.insert(desc)
params = {}
params['desc'] = desc
params['backtrace'] = BlackStack::API.backtrace
ret = BlackStack::API.post(
endpoint: "#{self.object_name}/insert",
params: params
)
raise "Error calling insert endpoint: #{ret['status']}" if ret['status'] != 'success'
return self.new(ret['result']).child_class_instance
end # def self.insert
# Submit a hash descriptor to the server for an upsert
#
def self.upsert(desc)
params = {}
params['desc'] = desc
params['backtrace'] = BlackStack::API.backtrace
ret = BlackStack::API.post(
endpoint: "#{self.object_name}/upsert",
params: params
)
raise "Error calling upsert endpoint: #{ret['status']}" if ret['status'] != 'success'
return ret['result'] == {} ? nil : self.new(ret['result']).child_class_instance
end # def self.upsert
# Submit a hash descriptor to the server for an upsert
#
def upsert
self.class.upsert(self.desc)
end
# return the HTML of a page downloaded by Zyte.
#
# Parameters:
# - url: the URL of the page to download.
# - api_key: the Zyte API key.
# - options: the options to pass to Zyte.
#
def zyte_html(url, api_key:, options:, data_filename:)
ret = nil
# getting the HTML
zyte = ZyteClient.new(key: api_key)
html = zyte.extract(url: url, options: options, data_filename: data_filename)
# return the URL of the file in the cloud
return html
end # def zyte_html
# create a file in the cloud with the HTML of a page downloaded by Zyte.
# return the URL of the file.
#
# Parameters:
# - url: the URL of the page to download.
# - api_key: the Zyte API key.
# - options: the options to pass to Zyte.
# - dropbox_folder: the folder in the cloud where to store the file. If nil, it will use the self.desc['id_account'] value.
# - retry_times: the number of times to retry the download until the HTML is valid.
#
def zyte_snapshot(url, api_key:, options:, data_filename:, dropbox_folder:nil, retry_times: 3)
# "The garbage character must be due to the 520 error code which was caused on the second request."
garbage = "\x9E\xE9e"
ret = nil
raise "Either dropbox_folder parameter or self.desc['id_account'] are required." if dropbox_folder.nil? && self.desc['id_account'].nil?
dropbox_folder = self.desc['id_account'] if dropbox_folder.nil?
# build path to the local file in /tmp
id = SecureRandom.uuid
filename = "#{id}.html"
tmp_path = "/tmp/#{filename}"
# build path to the file in the cloud
year = Time.now.year.to_s.rjust(4,'0')
month = Time.now.month.to_s.rjust(2,'0')
dropbox_folder = "/massprospecting.bots/#{dropbox_folder}.#{year}.#{month}"
dropbox_path = "#{dropbox_folder}/#{filename}"
# getting the HTML - Retry mechanism
zyte = ZyteClient.new(key: api_key)
try = 0
html = garbage
while try < retry_times && html == garbage
html = zyte.extract(url: url, options: options, data_filename: data_filename)
try += 1
end
# save the HTML in the local file in /tmp
File.open(tmp_path, 'w') { |file| file.write(html) }
# create the folder in the cloud and upload the file
BlackStack::DropBox.dropbox_create_folder(dropbox_folder)
BlackStack::DropBox.dropbox_upload_file(tmp_path, dropbox_path)
# delete the local file
File.delete(tmp_path)
# return the URL of the file in the cloud
return BlackStack::DropBox.get_file_url(dropbox_path)
end # def zyte_snapshot
end # class Base
module BlackStack
Skeleton classes use to extend these Ruby modules:
# MassProspecting Configurations
module Mass
# default domain for tracking
@@tracking_url = nil
# merge-tags
UNSUBSCRIBE_MERGETAG = '{unsubscribe-url}'
@@mergetags = [
'{company-name}',
#'{company-location}',
#'{company-industry}',
#'{company-email}',
#'{company-phone}',
#'{company-linkedin}',
#'{company-facebook}',
'{first-name}',
'{last-name}',
'{job-title}',
'{location}',
'{email}',
'{phone}',
'{linkedin}',
'{facebook}',
UNSUBSCRIBE_MERGETAG,
'{unsubscribe-link}',
]
def self.set_mergetags(tags)
@@mergetags = tags
end
def self.mergetags
@@mergetags
end
# MassProspecting API
def self.set(
tracking_url:'http://127.0.0.1:3000'
)
@@tracking_url = tracking_url
end
def self.tracking_url
@@tracking_url
end
module Constants
URL_PATTERN = /^http(s)?:\/\/[a-z0-9\-\._~:\/\?#\[\]@!\$&'\(\)\*\+,;=]+$/i
MAX_COLOR_VALUE = 16777215
MAX_INT = 2147483647
MAX_PORT_NUMBER = 65535
MAX_BIGINT = 36550000000000
end # module Constants
module State
# return an array with the different type of states
def states
[:idle, :online, :starting, :stopping, :scraping_inbox, :scraping_connections, :scraping, :enrichment, :outreach]
end # def self.states
# return an array with the color of differy type of states
def state_colors
[:black, :green, :cyan, :pink, :orange, :blue, :orange, :orange, :orange]
end # def self.state_colors
# return position of the array self.states, with the key.to_sym value
def state_code(key)
self.states.index(key.to_sym)
end # def self.state_code
end # module State
module VerificationResult
def self.results()
[
:any,
:pending, # this value means that the verification has not been processed yet.
:valid,
:invalid,
:catchall,
:role,
:unknown,
:disabled,
:disposable,
:inbox_full,
:role_account,
:spamtrap,
]
end
def results
Mass::VerificationResult.results
end
def result_code(key)
self.results.index(key.to_sym)
end
# validate the values of some specific keys are valid tristates.
def verification_email_result_errors(h={}, keys:)
ret = []
keys.each do |k|
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a valid tristate (#{Mass::VerificationResult.results.join(",")})." if h[k.to_s] && Mass::VerificationResult.results.index(h[k.to_s].to_sym).nil?
end # keys.each
return ret
end # def linkedin_errors
end # module VerificationResult
module Access
# return an array with the different type of accesses
def accesses
[:rpa, :api, :mta, :basic]
end # def self.accesses
# return an array with the different type of accesses
def access_colors
[:cyan, :orange, :purple, :gray]
end # def self.accesses
# return position of the array self.accesses, with the key.to_sym value
def access_code(key)
self.accesses.index(key.to_sym)
end # def self.access
end # module Access
module Direction
# return an array with the different type of directions
def directions
[:outgoing, :incoming, :accepted]
end # def self.accesses
# return position of the array self.states, with the key.to_sym value
def direction_code(key)
self.directions.index(key.to_sym)
end # def self.access
end # module Direction
module MTA
def authentications
[:plain, :login, :cram_md5, :none]
end
def authentication_code(key)
self.authentications.index(key.to_sym)
end
# OpenSSL::SSL::VERIFY_NONE == 0
# OpenSSL::SSL::VERIFY_PEER == 1
# OpenSSL::SSL::VERIFY_FAIL_IF_NO_PEER_CERT == 2
# OpenSSL::SSL::VERIFY_CLIENT_ONCE == 4
def openssl_verify_modes
[:none, :peer, :fail_if_no_peer_cert, nil, :client_once]
end
def openssl_verify_mode_code(key)
self.openssl_verify_modes.index(key.to_sym) if key
end
end
module Body
# return an array with the different type of bodies
def bodies
[:plain, :html]
end # def self.bodies
# return position of the array self.accesses, with the key.to_sym value
def body_code(key)
self.bodies.index(key.to_sym)
end # def self.body_code
end # module Body
module Type
# return an array with the different type of bodies
def types
[
:string,
:integer,
:float,
:boolean,
:date,
:time,
:text
]
end # def self.types
# return position of the array self.accesses, with the key.to_sym value
def type_code(key)
self.types.index(key.to_sym)
end # def self.type_code
end # module Type
module Triggerable
# return an array with the different type of triggers
def triggers
[
:rule_performed,
:event_created,
:tag_added,
:enrichment_done,
:outreach_done, # outgoing messages only
:accepted_request,
:incoming_message,
:open,
:click,
]
end # def self.triggers
# return position of the array self.triggers, with the key.to_sym value
def trigger_code(key)
self.triggers.index(key.to_sym)
end # def self.triggers
end # module Triggerable
module Parameters
# return an array with the different type of parameters (aka: on-the-fly filters)
# that can be supported by an outreach or enrichment either.
def parameters
[
'email_verification_result',
'email',
'phone',
'linkedin',
'facebook',
'english_names_only',
'keyword',
'industry',
'headcount',
'revenue',
'country',
'sic',
'naics',
'location',
]
end # def self.filters
end
module Filterable
# return an array with the different type of filters
def filters
[
:lead,
:company,
:event,
:any
]
end # def self.filters
# return position of the array self.triggers, with the key.to_sym value
def filter_code(key)
self.filters.index(key.to_sym)
end # def self.filter_code
end # module Filterable
module Actionable
# return an array with the different type of actions
def actions
[
:new_enrichment,
:new_outreach,
:add_tag,
:remove_tag,
:ai,
]
end # def self.actions
# return position of the array self.actions, with the key.to_sym value
def action_code(key)
self.actions.index(key.to_sym)
end # def self.actions
end # module Actionable
module Unit
# return an array with the different type of states
def units
[:minutes, :hours, :days, :weeks, :months]
end # def self.units
# return position of the array self.states, with the key.to_sym value
def unit_code(key)
self.units.index(key.to_sym)
end # def self.unit_code
end # module Unit
module Skip
# return an array with the different type of states
def skips
[:global, :tag_only, :none]
end # def self.units
# return position of the array self.states, with the key.to_sym value
def skip_code(key)
self.skips.index(key.to_sym)
end # def self.unit_code
end # module Skip
module TimelinePendings
# Return up to `limit` objects `Mass::Job` with `job.creation_tracking_time` is `null`.
def pending_summarize_creation_to_timeline(limit:100)
return self.where(
Sequel.lit("
summarize_creation_to_timeline IS NULL
")
).order(:create_time).limit(limit).all
end
# Return up to `limit` objects `Mass::Job` with `job.status_tracking_time` is `null` and `job.done_time` is not `null`.
def pending_summarize_status_to_timeline(limit:100)
return self.where(
Sequel.lit("
summarize_status_to_timeline IS NULL AND
done_time IS NOT NULL AND
\"status\" IN (
#{Mass::Outreach.status_code(:performed)},
#{Mass::Outreach.status_code(:failed)},
#{Mass::Outreach.status_code(:aborted)}
)
")
).order(:done_time).limit(limit).all
end
end # module TimelinePendings
end # module Mass
module BlackStack
module InsertUpdate
# return a Sequel dataset, based on some filters.
# this method is used by the API to get the data from the database remotely
def base_list(account=nil, filters: {})
ds = nil
if account
ds = self.where(:id_account => account.id, :delete_time => nil)
else
ds = self.where(:delete_time => nil)
end
return ds
end
# return a Sequel dataset, based on some filters and some pagination parameters.
# this method is used by the API to get the data from the database remotely
def page(account=nil, page:, limit:, filters: {})
ds = self.list(account, filters: filters)
ds = ds.limit(limit).offset((page-1)*limit).order(:create_time)
return ds
end
# insert a record
#
# parameters:
# - upsert_children: call upsert on children objects, or call insert on children objects
#
# return the object
def insert(h={}, upsert_children: true)
h = self.normalize(h)
errors = self.errors(h)
raise errors.join("\n") if errors.size > 0
b = upsert_children
o = self.new
o.id = h['id'] || guid
o.id_account = h['id_account']
o.id_user = h['id_user']
o.create_time = now
o.update(h, upsert_children: b)
o.save
return o
end
# insert or update a record
#
# parameters:
# - upsert_children: call upsert on children objects, or call insert on children objects
#
# return the object
def upsert(h={}, upsert_children: true)
o = self.find(h)
b = upsert_children
if o.nil?
o = self.insert(h, upsert_children: b)
else
o.update(h, upsert_children: b)
end
return o
end # def self.upsert
# insert or update an array of hash descriptors
def insert_update_many(a, logger: nil)
l = logger || BlackStack::DummyLogger.new
i = 0
a.each do |h|
i += 1
l.logs "Processing record #{i}/#{a.size}... "
self.upsert(h)
l.logf 'done'.green
end
end # def insert_update_many
end
module Serialize
# update a record
#
# parameters:
# - upsert_children: call upsert on children objects, or call insert on children objects.
# - manage_done_time: if this is moving fom the status :pending to another status at first time, set the done_time.
#
# return the object
def base_update(h={}, upsert_children: true, manage_done_time: false)
h = self.class.normalize(h)
errors = self.class.errors(h)
raise errors.join("\n") if errors.size > 0
# if this is moving fom the status :pending to another status at first time, set the done_time
if manage_done_time
if self.status == self.class.status_code(:pending) && !h['status'].to_s.empty? && h['status'].to_sym != :pending && self.done_time.nil?
self.done_time = now
end
end
return self
end
# call the method upsert for each element of the array a.
def to_h_base
{
'id' => self.id,
'id_account' => self.id_account,
'id_user' => self.id_user,
'create_time' => self.create_time,
'update_time' => self.update_time,
'delete_time' => self.delete_time,
}
end # def to_h
def save
self.update_time = now
super
end
end
module Palette
# parameter rgb is an array of 3 integers, each one between 0 and 255.
# return an integer with the RGB value.
def rgb_to_int(rgb)
ret = 0
ret += rgb[0] << 16
ret += rgb[1] << 8
ret += rgb[2]
return ret
end # def rgb_to_int
# parameter rgb is an array of 3 integers, each one between 0 and 255.
# return the HEX for the RGB value.
def rgb_to_hex(rgb)
ret = "#"
rgb.each do |c|
ret += c.to_s(16).rjust(2, '0')
end
return ret
end # def rgb_to_hex
# parameter n is an integer between 0 and 16777215.
# return an array of 3 integers, each one between 0 and 255.
def int_to_rgb(n)
ret = []
ret[0] = (n >> 16) & 0xFF
ret[1] = (n >> 8) & 0xFF
ret[2] = n & 0xFF
return ret
end # def int_to_rgb
# parameter n is an integer between 0 and 16777215.
# return the HEX for the RGB value.
def int_to_hex(n)
rgb = self.int_to_rgb(n)
return self.rgb_to_hex(rgb)
end
# Return a hash with 25 RGB values
# Each color must support black text inside.
# So, avoid dark colors.
def pallette
{
:red => [255, 0, 0],
:green => [0, 255, 0],
:blue => [0, 0, 255],
:yellow => [255, 255, 0],
:orange => [255, 165, 0],
:cyan => [0, 255, 255],
:magenta => [255, 0, 255],
:white => [255, 255, 255],
:gray => [128, 128, 128],
:maroon => [128, 0, 0],
:olive => [128, 128, 0],
:navy => [0, 0, 128],
:purple => [128, 0, 128],
:teal => [0, 128, 128],
:light_blue => [173, 216, 230],
:light_green => [144, 238, 144],
:light_red => [255, 182, 193],
:black => [0, 0, 0],
}
end # def pallette
# parameter key is a symbol
# return the HEX for the RGB value defined in the palette.
def pallette_to_hex(key)
rgb = self.pallette[key]
return self.rgb_to_hex(rgb)
end # def pallette_to_hex
# parameter key is a symbol
# return the RGB value defined in the palette.
def pallette_to_rgb(key)
return self.pallette[key]
end # def pallette_to_rgb
# parameter key is a symbol
# return the RGB value defined in the palette.
def pallette_to_int(key)
rgb = self.pallette[key]
return self.rgb_to_int(rgb)
end # def pallette_to_int
end # module Palette
module Storage
# Download a file form the web and store it in Dropbox.
# Return the URL of the file in Dropbox.
#
# Parameters:
# - url: the URL of the file to download.
# - filename: the name of the file to store in Dropbox.
# - dropbox_folder: the name of the folder in Dropbox where the file will be stored. E.g.: "channels/#{account.id}".
# - downloadeable: if true, the returnled URL will allow the download of the file. If false, the URL will show the file in the browser.
#
def store(url:, filename:, dropbox_folder:, downloadeable: false)
tempfile = nil
if url =~ /^http(s)?\:\/\//
tempfile = Down.download(url)
elsif url =~ /^file\:\/\//
tempfile = File.open(url.gsub(/^file\:\/\//, '/'))
end
year = Time.now.year.to_s.rjust(4,'0')
month = Time.now.month.to_s.rjust(2,'0')
folder = "/massprospecting.bots/#{dropbox_folder}.#{year}.#{month}"
path = "#{folder}/#{filename}"
BlackStack::DropBox.dropbox_create_folder(folder)
BlackStack::DropBox.dropbox_upload_file(tempfile.path, path)
ret = BlackStack::DropBox.get_file_url(path)
ret.gsub!(/\&dl\=1/, "&dl=0") if downloadeable == false
# delete the temporary file if I downloaded it form the web
if url =~ /^http(s)?\:\/\//
# delete the temporary file from the hard drive
File.delete(tempfile.path) if File.exists?(tempfile.path)
end
# return the URL of the file in Dropbox.
return ret
end
end # module Storage
module Status
# return an array with the different type of states
# - :pending - it is waiting to be executed
# - :running - it is being executed right now
# - :performed - it was executed successfully
# - :failed - it was executed with errors
# - :aborted - it started but it was aborted because filters didn't pass
# - :canceled - it was canceled by the user
#
def statuses
[:pending, :running, :performed, :failed, :aborted, :canceled]
end # def self.accesses
# return an array with the color of differy type of status
def status_colors
[:gray, :blue, :green, :red, :black, :black]
end # def self.state_colors
# return position of the array self.states, with the key.to_sym value
def status_code(key)
self.statuses.index(key.to_sym)
end # def self.access
end # module Status
module DomainProtocol
# return an array with the different type of domain_protocols
#
def domain_protocols
[:http, :https]
end # def self.domain_protocols
# return position of the array self.domain_protocols, with the key.to_sym value
def domain_protocols_code(key)
self.domain_protocols.index(key.to_sym)
end # def self.domain_protocols_code
end # module DomainProtocol
module Validation
INDEED_COMPANY_PATTERN = /^https\:\/\/(www\.)?indeed\.com\/cmp\//
# normalize the values of the descriptor.
def normalize(h={})
h['id_account'] = h['id_account'].to_s.strip.downcase if h['id_account']
h['id_user'] = h['id_user'].to_s.strip.downcase if h['id_user']
return h
end # def self.normalize
# return an array of error messages if there are keys now allowed
def key_errors(h={}, allowed_keys:)
allowed_keys += [:id_account, :id_user, :id, :create_time, :update_time, :delete_time]
ret = []
h.keys.each do |k|
ret << "The key '#{k}' is not allowed in #{self.name.gsub('Mass::', '')}." if !allowed_keys.map { |s| s.to_s }.include?(k.to_s)
end
return ret
end # def key_errors
# validate some keys of the hash descriptor are present
def mandatory_errors(h={}, keys:)
ret = []
keys.each do |k|
ret << "The #{k} is required for #{self.name.gsub('Mass::', '')}." if h[k.to_s].nil? || h[k.to_s].to_s.empty?
end
return ret
end # def mandatory_errors
# validate the values of some specific keys are valid URLs.
def url_errors(h={}, keys:)
ret = []
keys.each do |k|
unless h[k.to_s].nil? || h[k.to_s].to_s.empty?
# validate the URL is reachable
if h[k.to_s].to_s.valid_url?
begin
uri = URI.parse(h[k.to_s])
response = Net::HTTP.get_response(uri)
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} is not reachable." if response.code.to_i != 200
rescue
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} is not reachable."
end
elsif h[k.to_s].to_s.valid_filename?
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} is not reachable." if !File.exists?(h[k.to_s].gsub(/^file\:\/\//, '/'))
else
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} is not a valid URL or File."
end
end
end # keys.each
return ret
end # def url_errors
# validate the values of some specific keys are valid Emails.
def email_errors(h={}, keys:)
ret = []
keys.each do |k|
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a valid Email." if h[k.to_s] && !h[k.to_s].to_s.match(/^[\w\.\-]+@[\w\.\-]+\.\w+$/)
end # keys.each
return ret
end # def email_errors
def normalize_linkedin_url(url)
# remove query string parameters
url = url.gsub(/\?.*$/, '')
# remove # at the end
url = url.gsub(/\#.*$/, '')
# add 'http' at the begining if it doesn't exists
url = "http://#{url}" if !url.match(/^http(s)?\:\//)
# add 'www.' at the begining if it doesn't exists
url = url.gsub(/^http(s)?\:\/\/(www\.)?/, 'https://www.')
# remove the last '/' if it exists
url = url.gsub(/\/$/, '')
return url.to_s.downcase
end
# validate the values of some specific keys are valid Emails.
def linkedin_errors(h={}, keys:)
ret = []
keys.each do |k|
url = h[k.to_s].nil? ? nil : self.normalize_linkedin_url(h[k.to_s])
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a valid LinkedIn URL." if url && !url.to_s.match(/^http(s)?\:\/\/(www\.)?linkedin\.com\/in\/[^\/]+$/i)
end # keys.each
return ret
end # def linkedin_errors
def normalize_linkedin_company_url(url)
# remove query string parameters
url = url.gsub(/\?.*$/, '')
# remove # at the end
url = url.gsub(/\#.*$/, '')
# add 'http' at the begining if it doesn't exists
url = "http://#{url}" if !url.match(/^http(s)?\:\//)
# add 'www.' at the begining if it doesn't exists
url = url.gsub(/^http(s)?\:\/\/(www\.)?/, 'https://www.')
# remove the last '/' if it exists
url = url.gsub(/\/$/, '')
return url.to_s.downcase
end
# validate the values of some specific keys are valid Emails.
def linkedin_company_errors(h={}, keys:)
ret = []
keys.each do |k|
url = h[k.to_s].nil? ? nil : self.normalize_linkedin_company_url(h[k.to_s])
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a valid LinkedIn URL." if url && !url.to_s.match(/^http(s)?\:\/\/(www\.)?linkedin\.com\/company\/[^\/]+$/i)
end # keys.each
return ret
end # def linkedin_errors
def normalize_indeed_company_url(url)
url.split('?').first
end
# validate the values of some specific keys are valid Emails.
def indeed_company_errors(h={}, keys:)
ret = []
keys.each do |k|
url = h[k.to_s].nil? ? nil : self.normalize_indeed_company_url(h[k.to_s])
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a valid Indeed URL." if url && url.to_s !~ INDEED_COMPANY_PATTERN
end # keys.each
return ret
end # def linkedin_errors
# validate the values of some specific keys are valid tristates.
def tristate_errors(h={}, keys:)
ret = []
keys.each do |k|
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a valid tristate (#{BlackStack::Tristate.tristates.join(",")})." if h[k.to_s] && BlackStack::Tristate.tristates.index(h[k.to_s].to_sym).nil?
end # keys.each
return ret
end # def linkedin_errors
def normalize_facebook_url(url)
# remove query string parameters
url = url.gsub(/\?.*$/, '')
# remove # at the end
url = url.gsub(/\#.*$/, '')
# add 'http' at the begining if it doesn't exists
url = "http://#{url}" if !url.match(/^http(s)?\:\//)
# remove 'www.'
url = url.gsub(/^http(s)?\:\/\/(www\.)?/, 'https://')
# add 'www.' at the begining if it doesn't exist
url = url.gsub(/^http(s)?\:\/\/(web\.)?/, 'https://web.')
# remove /refname/
url = url.gsub(/\/refname\//, '/')
# remove the last '/' if it exists
url = url.gsub(/\/$/, '')
return url.to_s.downcase
end
# validate the values of some specific keys are valid Emails.
def facebook_errors(h={}, keys:)
ret = []
keys.each do |k|
url = h[k.to_s].nil? ? nil : self.normalize_facebook_url(h[k.to_s])
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a valid Facebook URL." if url && !url.to_s.match(/^http(s)?\:\/\/(web\.)?facebook\.com\/[a-z0-9\-\._]+$/i)
end # keys.each
return ret
end # def linkedin_errors
def normalize_facebook_company_url(url)
normalize_facebook_url(url)
end
# validate the values of some specific keys are valid Emails.
def facebook_company_errors(h={}, keys:)
facebook_errors(h, keys: keys)
end # def linkedin_errors
# get domain from url using URI
def normalize_email(email)
email.to_s.strip.downcase
end
# get domain from url using URI
def normalize_domain(url)
url = url.to_s.strip.downcase
# remove query string parameters
url = url.gsub(/\?.*$/, '')
# remove # at the end
url = url.gsub(/\#.*$/, '')
# add 'http' at the begining if it doesn't exists
url = "http://#{url}" if !url.match(/^http(s)?\:\//)
# remove 'www.'
url = url.gsub(/^http(s)?\:\/\/(www\.)?/, '')
# add 'www.' at the begining if it doesn't exist
url = url.gsub(/^http(s)?\:\/\/(web\.)?/, '')
# remove the last '/' if it exists
url = url.gsub(/\/$/, '')
# remove protoco
url = url.gsub(/^http\:\/\//, '')
url = url.gsub(/^https\:\/\//, '')
return url.to_s.downcase
end
# validate the values of some specific keys are valid strings
def string_errors(h={}, keys:)
ret = []
keys.each do |k|
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a string." if h[k.to_s] && !h[k.to_s].is_a?(String)
end
return ret
end # def string_errors
# validate the values of some specific keys are valid domains
def domain_errors(h={}, keys:)
ret = []
keys.each do |k|
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a valid domain." if h[k.to_s] && !h[k.to_s].to_s.strip.downcase.valid_domain?
end
return ret
end # def domain_errors
# validate the values of some specific keys are valid integers, between 2 values.
def int_errors(h={}, min:, max:, keys:)
ret = []
keys.each do |k|
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be an integer." if h[k.to_s] && !h[k.to_s].is_a?(Integer)
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be greater than #{min}." if h[k.to_s] && h[k.to_s].is_a?(Integer) && h[k.to_s] < min
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be less than #{max}." if h[k.to_s] && h[k.to_s].is_a?(Integer) && h[k.to_s] > max
end
return ret
end # def int_errors
# validate the values of some specific keys are valid floats, between 2 values.
def float_errors(h={}, min:, max:, keys:)
ret = []
keys.each do |k|
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a float." if h[k.to_s] && !h[k.to_s].is_a?(Float)
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be greater than #{min}." if h[k.to_s] && h[k.to_s].is_a?(Float) && h[k.to_s] < min
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be less than #{max}." if h[k.to_s] && h[k.to_s].is_a?(Float) && h[k.to_s] > max
end
return ret
end # def float_errors
# validate the values of some specific keys are valid booleans.
def boolean_errors(h={}, keys:)
ret = []
keys.each do |k|
ret << "The #{k} '#{h[k.to_s].to_s}' for #{self.name.gsub('Mass::', '')} must be a boolean." if h[k.to_s] && ![true, false].include?(h[k.to_s])
end
return ret
end # def boolean_errors
# return an array of error messages regarding the ownership of the object.
def ownership_errors(h={})
ret = []
ret << "The id_account for #{self.name.gsub('Mass::', '')} is required." if h['id_account'].nil? || h['id_account'].to_s.empty?
ret << "The id_account for #{self.name.gsub('Mass::', '')} must be a guid." if h['id_account'] && !h['id_account'].to_s.guid?
ret << "The id_account for #{self.name.gsub('Mass::', '')} must exist." if h['id_account'] && h['id_account'].to_s.guid? && BlackStack::MySaaS::Account.where(:id => h['id_account']).first.nil?
ret << "The id_user for #{self.name.gsub('Mass::', '')} must be a guid." if h['id_user'] && !h['id_user'].to_s.guid?
ret << "The id_user for #{self.name.gsub('Mass::', '')} must exist." if h['id_user'] && h['id_user'].to_s.guid? && BlackStack::MySaaS::User.where(:id => h['id_user']).first.nil?
if h['id_account'] && h['id_user']
# user must be belonging the account
ret << "The user for #{self.name.gsub('Mass::', '')} must belong to the account." if BlackStack::MySaaS::User.where(:id => h['id_user'], :id_account => h['id_account']).first.nil?
end
return ret
end # def ownership_errors
# return an array of error messages regarding the naming of the object.
# validations:
# - name is required.
# - the name must be unique.
# - the name must be a string.
# - the name must be less than 255 characters.
#
def naming_errors(h={})
h = self.normalize(h)
ret = []
ret << "The name for #{self.name.gsub('Mass::', '')} is required." if h['name'].nil? || h['name'].to_s.empty?
ret << "The name for #{self.name.gsub('Mass::', '')} must be a string." if h['name'] && !h['name'].is_a?(String)
ret << "The name for #{self.name.gsub('Mass::', '')} must be less than 255 characters." if h['name'] && h['name'].is_a?(String) && h['name'].to_s.length > 255
=begin
# name must be unique
ret << "Name is unique and already exists a #{self.name.gsub('Mass::', '')} with the same name." if h['name'] && h['id'].nil? && self.where(
:id_account => h['id_account'],
:name => h['name'].to_s
).first
ret << "Name is unique and already exists a #{self.name.gsub('Mass::', '')} with the same name." if h['name'] && h['id'] && self.where(Sequel.lit("
id_account = '#{h['id_account']}' AND
name = '#{h['name'].to_s.to_sql}' AND
id != '#{h['id']}'
")).first
=end
return ret
end # def naming_errors
# return an array of error messages regarding the color of the object.
def color_errors(h={})
ret = []
ret << "The color_code for #{self.name.gsub('Mass::', '')} must be a symbol or string." if h['color_code'] && !h['color_code'].is_a?(Symbol) && !h['color_code'].is_a?(String)
ret << "The color_code for #{self.name.gsub('Mass::', '')} must be a valid key from the palette." if h['color_code'] && h['color_code'].is_a?(Symbol) && !pallette.keys.include?(h['color_code'].to_sym)
return ret
end # def color_errors
end # module Validation
module Tristate
def self.tristates
[:any, :yes, :no]
end
def tristates
BlackStack::Tristate.tristates
end
def tristate_code(key)
self.tristates.index(key.to_sym)
end
end
end ## module BlackStack
Training Prompt 2
The communication between a stub instances and its counter-part skeleton instance is trough the `Mass::Client** library.
mass-client.rb:
require 'uri'
require 'net/http'
require 'json'
require 'blackstack-core'
require 'simple_cloud_logging'
require 'simple_command_line_parser'
require 'colorize'
require 'timeout'
require 'base64'
require 'adspower-client'
require 'aws-sdk-s3' # Ensure the AWS SDK for S3 is installed
# mass client configuration
module Mass
@@js_path
@@drownload_path
@@api_client
@@s3_bucket
@@s3_region
@@s3_access_key_id
@@s3_secret_access_key
@@s3
# set the MassProspecting API client
#
# Parameters:
#
# api_key: Mandatory. The API key of your MassProspecting account.
# subaccount: Optional. The name of the subaccount you want to work with. If you provide a subaccount, the SDK will call the master to get the URL and port of the subaccount. Default is nil.
#
# api_url: Optional. The URL of the MassProspecting API. Default is 'https://massprospecting.com'.
# api_port: Optional. The port of the MassProspecting API. Default is 443.
# api_version: Optional. The version of the MassProspecting API. Default is '1.0'.
#
# backtrace: Optional. If true, the backtrace of the exceptions will be returned by the access points. If false, only an error description is returned. Default is false.
#
# js_path: Optional. The path to the JavaScript file to be used by the SDK. Default is nil.
# download_path: Optional. The path to the download folder(s) to be used by the SDK. Default is [].
#
# s3_region, s3_access_key_id, s3_secret_access_key, s3_bucket: Defining AWS S3 parameter for storing files at the client-side.
#
def self.set(
api_key: ,
subaccount: nil,
api_url: 'https://massprospecting.com',
api_port: 443,
api_version: '1.0',
backtrace: false,
js_path: nil,
download_path: [],
s3_region: nil,
s3_access_key_id: nil,
s3_secret_access_key: nil,
s3_bucket: nil
)
# call the master to get the URL and port of the subaccount.
BlackStack::API.set_client(
api_key: api_key,
api_url: api_url,
api_port: api_port,
api_version: api_version,
backtrace: backtrace
)
if subaccount
params = { 'name' => subaccount }
ret = BlackStack::API.post(
endpoint: "resolve/get",
params: params
)
raise "Error initializing client: #{ret['status']}" if ret['status'] != 'success'
# call the master to get the URL and port of the subaccount.
BlackStack::API.set_client(
api_key: ret['api_key'],
api_url: ret['url'],
api_port: ret['port'],
api_version: api_version,
backtrace: backtrace
)
end
# validate: download_path must be a string or an arrow of strings
if download_path.is_a?(String)
raise ArgumentError.new("The parameter 'download_path' must be a string or an array of strings.") if download_path.to_s.empty?
elsif download_path.is_a?(Array)
download_path.each { |p|
raise ArgumentError.new("The parameter 'download_path' must be a string or an array of strings.") if p.to_s.empty?
}
else
raise ArgumentError.new("The parameter 'download_path' must be a string or an array of strings.")
end
@@js_path = js_path
@@download_path = download_path
@@s3_region = s3_region
@@s3_access_key_id = s3_access_key_id
@@s3_secret_access_key = s3_secret_access_key
@@s3_bucket = s3_bucket
# Initialize the S3 client
if (
@@s3_region
@@s3_access_key_id
@@s3_secret_access_key
@@s3_bucket
)
@@s3 = Aws::S3::Client.new(
region: @@s3_region,
access_key_id: @@s3_access_key_id,
secret_access_key: @@s3_secret_access_key
)
end
end
def self.download_path
@@download_path
end
def self.js_path
@@js_path
end
def self.s3_region
@@s3_region
end
def self.s3_access_key_id
@@s3_access_key_id
end
def self.s3_secret_access_key
@@s3_secret_access_key
end
def self.s3_bucket
@@s3_bucket
end
def self.s3
@@s3
end
end # module Mass
# base classes
require_relative './/base-line/channel'
require_relative './/base-line/profile_type'
require_relative './/base-line/source_type'
require_relative './/base-line/enrichment_type'
require_relative './/base-line/outreach_type'
require_relative './/base-line/data_type'
require_relative './/base-line/headcount'
require_relative './/base-line/industry'
require_relative './/base-line/location'
require_relative './/base-line/revenue'
require_relative './/base-line/tag'
require_relative './/base-line/profile'
require_relative './/base-line/source'
require_relative './/base-line/job'
require_relative './/base-line/event'
require_relative './/base-line/outreach'
require_relative './/base-line/enrichment'
require_relative './/base-line/unsubscribe'
require_relative './/base-line/company'
#require_relative './/base-line/company_data'
#require_relative './/base-line/company_industry'
#require_relative './/base-line/company_naics'
#require_relative './/base-line/company_sic'
#require_relative './/base-line/company_tag'
require_relative './/base-line/lead'
#require_relative './/base-line/lead_data'
#require_relative './/base-line/lead_tag'
require_relative './/base-line/inboxcheck'
require_relative './/base-line/connectioncheck'
require_relative './/base-line/rule'
require_relative './/base-line/request'
# first line of children
require_relative './/first-line/profile_api'
require_relative './/first-line/profile_mta'
require_relative './/first-line/profile_rpa'
Training Prompt 3
I am writing a copilot to operate my SaaS:
mass-client.rb:
require 'mass-client' #, '~>1.0.28'
require 'openai' #, '~>6.3.1'
require 'pry' #, '~>0.14.1'
module Mass
class Copilot
@@openai_api_key = nil
@@openai_model = nil
@@openai_client = nil
@@openai_assistant_id = nil
@@openai_thread_id = nil
@@openai_message_ids = []
# Create a new Copilot instance.
def initialize(openai_api_key:)
@@openai_api_key = openai_api_key
@@openai_client = OpenAI::Client.new(access_token: @@openai_api_key)
end
# return array of available models.
def models
response = @@openai_client.models.list
models = response['data'].map { |model| model['id'] }
models
end
# start a new conversation (thread), and return the ID of such a new thread.
def start(
openai_model: ,
instructions: ''
)
@@openai_model = openai_model
response = @@openai_client.assistants.create(
parameters: {
model: @@openai_model, # Retrieve via client.models.list. Assistants need 'gpt-3.5-turbo-1106' or later.
name: "OpenAI-Ruby test assistant",
description: nil,
instructions: instructions,
tools: [
{"type": "code_interpreter"},
{"type": "retrieval"},
## Tag Operations
{
type: "function",
function: {
name: "Mass::Tag.page",
description: "Get list of tags.",
parameters: {
type: :object,
properties: {
page: {
type: :integer,
description: "The number of page to return.",
},
limit: {
type: :integer,
description: "How many records per tag should retrieve.",
},
filters: {
type: :object,
description: "Filter parameters.",
properties: {
name: {
type: :string,
description: "Filter tags by name using partial match.",
},
},
},
},
required: ["command"],
},
},
},
],
metadata: { my_internal_version_id: '1.0.0' },
}
)
@@openai_assistant_id = response["id"]
# Create thread
response = @@openai_client.threads.create # Note: Once you create a thread, there is no way to list it
# or recover it currently (as of 2023-12-10). So hold onto the `id`
@@openai_thread_id = response["id"]
end
# for internal use only
def chat(prompt)
mid = @@openai_client.messages.create(
'thread_id': @@openai_thread_id,
'parameters': {
'role': "user", # Required for manually created messages
'content': prompt, # Required.
#'image_url': 'https://static.vecteezy.com/system/resources/thumbnails/039/391/155/small/500-social-media-stories-v2-abstract-shopping-nft-e-commerce-memories-bundle.png',
},
)["id"]
@@openai_message_ids << mid
# run the assistant
response = @@openai_client.runs.create(
thread_id: @@openai_thread_id,
parameters: {
assistant_id: @@openai_assistant_id,
}
)
run_id = response['id']
# wait for a response
while true do
response = @@openai_client.runs.retrieve(id: run_id, thread_id: @@openai_thread_id)
status = response['status']
case status
when 'queued', 'in_progress', 'cancelling'
sleep 1 # Wait one second and poll again
when 'completed'
break # Exit loop and report result to user
when 'requires_action'
tools_to_call = response.dig('required_action', 'submit_tool_outputs', 'tool_calls')
my_tool_outputs = tools_to_call.map { |tool|
# Call the functions based on the tool's name
function_name = tool.dig('function', 'name')
arguments = JSON.parse(
tool.dig("function", "arguments"),
{ symbolize_names: true },
)
begin
tool_output = case function_name
when "Mass::Tag.page"
Mass::Tag.page(**arguments)
else
raise "Unknown function name: #{function_name}"
end
{ tool_call_id: tool['id'], output: tool_output.to_s }
rescue => e
{ tool_call_id: tool['id'], output: "Error: #{e.message}" }
end
}
@@openai_client.runs.submit_tool_outputs(
thread_id: @@openai_thread_id,
run_id: run_id,
parameters: { tool_outputs: my_tool_outputs }
)
when 'cancelled', 'failed', 'expired'
raise response['last_error']['message']
break # or `exit`
else
raise "Unknown run status response from OpenAI: #{status}"
end
end
#
messages = @@openai_client.messages.list(thread_id: @@openai_thread_id)
messages['data'].first['content'].first['text']['value']
end # def chat
def console
puts "Mass-Copilot Console".blue
puts "Type 'exit' to quit.".blue
while true
print "You: ".green
prompt = gets.chomp
break if prompt == 'exit'
begin
puts "Mass-Copilot: #{chat(prompt)}".blue
rescue => e
puts "Error: #{e.message}".red
puts e.to_console
end
end
end # def console
end # class Copilot
end # module Mass
Requirement Prompt Example
This is the stub class Mass::Outreach
module Mass
class Outreach < BlackStack::Base
attr_accessor :type, :profile, :lead, :company, :profile_type
def self.object_name
'outreach'
end
def initialize(h)
super(h)
self.type = Mass::OutreachType.new(h['outreach_type_desc']) if h['outreach_type_desc']
self.profile = Mass::Profile.new(h['profile']).child_class_instance if h['profile']
self.lead = Mass::Lead.new(h['lead']) if h['lead']
self.company = Mass::Company.new(h['company']) if h['company']
self.profile_type = Mass::ProfileType.page(
id_account: h['id_account'],
page: 1,
limit: 1,
filters: {
name: h['profile_type']
}
).first.child_class_instance if h['profile_type']
end
# convert the outreach_type into the ruby class to create an instance.
# example: Apollo --> Mass::ApolloAPI
def class_name_from_outreach_type
outreach_type = self.desc['outreach_type']
"Mass::#{outreach_type}"
end
# crate an instance of the profile type using the class defined in the `desc['name']` attribute.
# override the base method
def child_class_instance
outreach_type = self.desc['outreach_type']
key = self.class_name_from_outreach_type
raise "Source code of outreach type #{outreach_type} not found. Create a class #{key} in the folder `/lib` of your mass-sdk." unless Kernel.const_defined?(key)
ret = Kernel.const_get(key).new(self.desc)
return ret
end
end # class Outreach
end # module Mass
This is the skeleton class Mass::Outreach
module Mass
class Outreach < Sequel::Model(:outreach)
extend Mass::State # make its methods as class methods
extend BlackStack::Status # make its methods as class methods
include BlackStack::Serialize # make its methods as class methods
extend BlackStack::Validation
extend Mass::Body # make its methods as class methods
extend Mass::Direction # make its methods as class methods
extend BlackStack::InsertUpdate # make its methods as class methods
extend Mass::TimelinePendings
extend Mass::Skip
many_to_one :account, :class => 'BlackStack::MySaaS::Account', :key => :id_account
many_to_one :user, :class => 'BlackStack::MySaaS::User', :key => :id_user
many_to_one :profile, :class => 'Mass::Profile', :key => :id_profile
many_to_one :outreach_type, :class => 'Mass::OutreachType', :key => :id_outreach_type
many_to_one :lead, :class => 'Mass::Lead', :key => :id_lead
many_to_one :company, :class => 'Mass::Company', :key => :id_company
many_to_one :tag, :class => 'Mass::Tag', :key => :id_tag
# track an outreach has been created by an action.
many_to_one :action, :class => 'Mass::Action', :key => :id_action
# Cancels all pending outreaches linked to paused rules.
#
# This method identifies all outreach records that are in the 'pending' state and linked to rules that have been paused.
# It then updates their status to 'canceled' in batches, to ensure efficient processing and minimize database load.
#
# @param batch_size [Integer] The number of records to process in each batch. Default is 100.
# @param logger [Object] A logger instance to log the progress of the operation. Defaults to BlackStack::DummyLogger if not provided.
#
# @example
# # Cancel all pending outreaches linked to paused rules with the default batch size and no custom logger.
# Outreach.cancel
#
# @example
# # Cancel all pending outreaches with a batch size of 50 and a custom logger.
# Outreach.cancel(batch_size: 50, logger: custom_logger)
#
# @return [void]
#
# The method iterates over the list of pending outreaches in batches. Each batch is then updated in bulk to
# improve efficiency, and logging is used to track the progress of each batch processed.
#
def self.cancel(
batch_size:100,
logger:nil
)
l = logger || BlackStack::DummyLogger.new(nil)
# Constants for status codes
pending_status = Mass::Outreach.status_code(:pending)
canceled_status = Mass::Outreach.status_code(:canceled)
# Define the batch size
batch_size = 100
# Fetch the list of pending outreaches linked to paused rules
l.logs "Get pending outreaches linked to paused rules... "
pending_outreaches = DB[:rule].join(
:rule_instance, id_rule: :id
).join(
:outreach, id_rule_instance: :id
).where(
active: false
).where(
Sequel[:outreach][:status] => pending_status
).select(Sequel[:outreach][:id])
l.done(details: pending_outreaches.count.to_s.blue)
# Process in batches of 100
i = 0
pending_outreaches.paged_each(
rows_per_fetch: batch_size
) do |outreach|
i += 1
l.logs "Cancel pending outreaches linked to paused rules (batch #{i.to_s.blue})... "
# Collect outreach IDs in the current batch
outreach_ids = []
outreach_ids << outreach[:id]
# Update the status of the collected outreach records to canceled
DB[:outreach].where(id: outreach_ids).update(status: canceled_status)
l.done
end
end # def self.cancel
# Reactivates all canceled outreaches linked to active rules.
#
# This method identifies all outreach records that are in the 'canceled' state and linked to rules that are currently active.
# It then updates their status to 'pending' in batches to ensure efficient processing and minimize database load.
#
# @param batch_size [Integer] The number of records to process in each batch. Default is 100.
# @param logger [Object] A logger instance to log the progress of the operation. Defaults to BlackStack::DummyLogger if not provided.
#
# @example
# # Reactivate all canceled outreaches linked to active rules with the default batch size and no custom logger.
# Outreach.reactivate
#
# @example
# # Reactivate all canceled outreaches with a batch size of 50 and a custom logger.
# Outreach.reactivate(batch_size: 50, logger: custom_logger)
#
# @return [void]
#
# The method iterates over the list of canceled outreaches in batches, reactivating each batch to a 'pending' status.
# Batch processing ensures database efficiency, and each batch is logged for easier monitoring and troubleshooting.
#
def self.reactivate(
batch_size: 100,
logger: nil
)
l = logger || BlackStack::DummyLogger.new(nil)
# Constants for status codes
canceled_status = Mass::Outreach.status_code(:canceled)
pending_status = Mass::Outreach.status_code(:pending)
# Fetch the list of canceled outreaches linked to active rules
l.logs "Get canceled outreaches linked to active rules... "
canceled_outreaches = DB[:rule].join(
:rule_instance, id_rule: :id
).join(
:outreach, id_rule_instance: :id
).where(
active: true
).where(
Sequel[:outreach][:status] => canceled_status
).select(Sequel[:outreach][:id])
l.done(details: canceled_outreaches.count.to_s.blue)
# Process in batches of specified size
i = 0
outreach_ids = [] # Collect outreach IDs for bulk updates
canceled_outreaches.paged_each(
rows_per_fetch: batch_size
) do |outreach|
outreach_ids << outreach[:id]
# If the batch is full or we are at the end, perform the update
if outreach_ids.size >= batch_size
i += 1
l.logs "Reactivate canceled outreaches linked to active rules (batch #{i.to_s.blue})... "
# Update the status of the collected outreach records to pending
DB[:outreach].where(id: outreach_ids).update(status: pending_status)
l.done
# Clear the batch
outreach_ids.clear
end
end
# Update any remaining records not included in a full batch
unless outreach_ids.empty?
i += 1
l.logs "Reactivate remaining canceled outreaches (batch #{i.to_s.blue})... "
# Update the status of the remaining outreach records to pending
DB[:outreach].where(id: outreach_ids).update(status: pending_status)
l.done
end
end
# Unassigns all pending outreaches linked to deleted profiles.
#
# This method identifies all outreach records that are in the 'pending' state and linked to profiles that have been deleted.
# It then removes the profile assignment to ensure data integrity.
#
# @param batch_size [Integer] The number of records to process in each batch. Default is 100.
# @param logger [Object] A logger instance to log the progress of the operation. Defaults to BlackStack::DummyLogger if not provided.
#
# @example
# # Unassign all pending outreaches linked to deleted profiles with the default batch size and no custom logger.
# Outreach.unassign
#
# @example
# # Unassign all pending outreaches with a batch size of 50 and a custom logger.
# Outreach.unassign(batch_size: 50, logger: custom_logger)
#
# @return [void]
#
def self.unassign(
batch_size: 100,
logger: nil
)
l = logger || BlackStack::DummyLogger.new(nil)
# Status code for pending outreaches
pending_status = Mass::Outreach.status_code(:pending)
# Fetch the list of pending outreaches linked to deleted profiles
l.logs "Get pending outreaches linked to deleted profiles... "
pending_outreaches = DB[:outreach].join(
:profile, Sequel[:outreach][:id_profile] => Sequel[:profile][:id]
).where(
Sequel[:outreach][:status] => pending_status
).where(
Sequel[:profile][:delete_time] => Sequel::NOTNULL
).select(Sequel[:outreach][:id])
l.done(details: pending_outreaches.count.to_s.blue)
# Process in batches of specified size
i = 0
outreach_ids = [] # Collect outreach IDs for bulk updates
pending_outreaches.paged_each(
rows_per_fetch: batch_size
) do |outreach|
outreach_ids << outreach[:id]
# If the batch is full or we are at the end, perform the unassignment
if outreach_ids.size >= batch_size
i += 1
l.logs "Unassign pending outreaches linked to deleted profiles (batch #{i.to_s.blue})... "
# Update the outreach records to unassign profiles (setting id_profile to nil)
DB[:outreach].where(id: outreach_ids).update(id_profile: nil)
l.done
# Clear the batch
outreach_ids.clear
end
end
# Update any remaining records not included in a full batch
unless outreach_ids.empty?
i += 1
l.logs "Unassign remaining pending outreaches (batch #{i.to_s.blue})... "
# Update the outreach records to unassign profiles
DB[:outreach].where(id: outreach_ids).update(id_profile: nil)
l.done
end
end
def basic_access?
self.outreach_type.basic_access?
end
# generate the CLI command to run this job
def command(url)
return nil if self.account.api_key.to_s.empty?
return nil if self.id_profile.nil? && !basic_access?
if self.basic_access?
return "ruby basic.rb " +
"api_key=#{self.account.api_key} " +
"api_url=#{url} " +
"ignore-hostname=yes " +
"force-profile-run=yes " +
"run-once=yes " +
"scraping=no " +
"enrichment=no " +
"outreach=yes " +
"id_outreach=#{self.id} "
else
return "ruby profile.rb " +
"api_key=#{self.account.api_key} " +
"api_url=#{url} " +
"id=#{self.id_profile} " +
"ignore-hostname=yes " +
"force-profile-run=yes " +
"headless=no " +
"run-once=yes " +
"inboxcheck=no " +
"connectioncheck=no " +
"scraping=no " +
"enrichment=no " +
"outreach=yes " +
"id_outreach=#{self.id} "
end
end
# if the access of the profile_type of the outrech_type of this outreach IS NOT :mta, then:
# - return the merged body.
#
# if the outreach direction is :outgoing, then:
# - return the body as is.
#
# if the access of the profile_type of the outrech_type of this outreach IS :mta, then:
# - return either html version of the body, assuming it is the full mime content.
# - reference: https://stackoverflow.com/questions/4868205/rails-mail-getting-the-body-as-plain-text
#
# TODO: Also, remove open tracking pixel
def simplified_body(preview: false)
# if the outreach direction is :outgoing, then:
if self.direction == self.class.direction_code(:outgoing)
return self.merged_body(preview: preview)
end
# if the access of the profile_type of the outrech_type of this outreach IS NOT :mta.
if self.outreach_type.profile_type.access != Mass::ProfileType.access_code(:mta)
return self.body
end
# if the access of the profile_type of the outrech_type of this outreach IS :mta.
ret = nil
# get the HTML body
m = Mail.new(self.body.to_s)
if m.html_part
ret = m.html_part.body.decoded
elsif m.body
ret = m.body.decoded
end
# even if I activated the preview flag I must remove the tracking pixel, links and unssubscribe link from quoted text.
# reference: https://github.com/ConnectionSphere/emails/issues/121
doc = Nokogiri::HTML(ret)
doc.css('style').remove # remove all css
doc.css('script').remove # remove all javascript
doc.xpath('//@style').remove # remove all inline css - https://stackoverflow.com/questions/6096327/strip-style-attributes-with-nokogiri
# remove tracking pixel
# reference: https://github.com/ConnectionSphere/emails/issues/99
doc.css('img').each do |img|
if img['src'] && img['src'].include?('/o/')
img.remove
end
end
# replace url of unsubscribe link
# reference: https://github.com/MassProspecting/docs/issues/165
doc.css('a').each do |a|
if a['href'] && a['href'].include?('/u/')
a['href'] = '#'
end
end
# replace url of tracking links by their original url
# reference: https://github.com/MassProspecting/docs/issues/165
doc.css('a').each do |a|
if a['href'] && a['href'].include?('/c/')
lid = a['href'].split('/').last
l = Mass::Link.where(:id => lid).first
if l
a['href'] = l.url
end
end
end
# get the processed body
body = doc.at('body')
if body
return body.inner_html
else
return doc.to_s
end
end # def simplified_body
# If text is nil, return nil.
# Generic method for internal use only.
# Replace merge-tags.
# Replace still peding merge-tags by their fallback spintax.
# Apply spintax.
#
# Parameters:
# - text: Content to replace merge-tags.
#
def generate_merged_text(text, preview: false)
return nil if text.nil?
raise "Outreach must have a profile assigned for merging." if self.profile.nil?
ret = text.dup
p = self.profile
# replace merge-tags
unsubscribe_url = '#'
if !preview
unsubscribe_url = "#{p.tracking_url}/u/#{self.id.to_guid}" if self.lead
unsubscribe_url = "#{p.tracking_url}/u/#{self.id.to_guid}" if self.company
end
ret.gsub!('{unsubscribe-url}', unsubscribe_url)
if self.action.nil?
ret.gsub!('{unsubscribe-link}', unsubscribe_url)
elsif self.action.body_type == Mass::Action.body_code(:html)
ret.gsub!('{unsubscribe-link}', "<a href='#{unsubscribe_url}' style='font-size:10px;'>Unsubscribe</a>")
else
ret.gsub!('{unsubscribe-link}', "unsubscribe: #{unsubscribe_url}")
end
#ret.gsub!(/\{company\-name[\&.*\;]*[\|[^\|^\}]*]*\}/, self.company.name) if self.company && self.company.name
ret.gsub!(/\{company\-name[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.company.name) if self.lead && self.lead.company && self.lead.company.name
ret.gsub!(/\{first\-name[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.first_name) if self.lead && self.lead.first_name
ret.gsub!(/\{last\-name[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.last_name) if self.lead && self.lead.last_name
ret.gsub!(/\{job\-title[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.job_title) if self.lead && self.lead.job_title
ret.gsub!(/\{location[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.location.name) if self.lead && self.lead.location && self.lead.location.name
ret.gsub!(/\{location[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.company.location.name) if self.lead && self.lead.company && self.lead.company.location && self.lead.company.location.name
#ret.gsub!(/\{company\-location[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.company.location.name) if self.company && self.company.location && self.company.location.name
#ret.gsub!(/\{company\-industry[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.company.company_industry.name) if self.company && self.company.company_industry.first && self.company.company_industry.first.industry.name
ret.gsub!(/\{email[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.email) if self.lead && self.lead.email
ret.gsub!(/\{email[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.company.email) if self.lead && self.lead.company && self.lead.company.email
#ret.gsub!(/\{company\-email[\&.*\;]*[\|[^\|^\}]*]*\}/, self.company.email) if self.company && self.company.email
ret.gsub!(/\{phone[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.phone) if self.lead && self.lead.phone
ret.gsub!(/\{phone[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.company.phone) if self.lead && self.lead.company && self.lead.company.phone
#ret.gsub!(/\{company\-phone[\&.*\;]*[\|[^\|^\}]*]*\}/, self.company.phone) if self.company && self.company.phone
ret.gsub!(/\{linkedin[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.linkedin) if self.lead && self.lead.linkedin
ret.gsub!(/\{linkedin[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.company.linkedin) if self.lead && self.lead.company && self.lead.company.linkedin
#ret.gsub!(/\{company\-linkedin[\&.*\;]*[\|[^\|^\}]*]*\}/, self.company.linkedin) if self.company && self.company.linkedin
ret.gsub!(/\{facebook[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.facebook) if self.lead && self.lead.facebook
ret.gsub!(/\{facebook[\&.*\;]*[\|[^\|^\}]*]*\}/, self.lead.company.facebook) if self.lead && self.lead.company && self.lead.company.facebook
#ret.gsub!(/\{company\-facebook[\&.*\;]*[\|[^\|^\}]*]*\}/, self.company.facebook) if self.company && self.company.facebook
# replace merge-tags with fallback values by the a spintax
# example: any substring like {company-name>an amazing company|a great company} is replaced by {an amazing company|a great company} by removing the {company-name> substring.
Mass::mergetags.each { |r|
# iterate all the substrings /\{first\-name[\&.*\;]*\|[^\}]*\}/
r.gsub!(/^{/, '')
r.gsub!(/}$/, '')
r = Regexp.escape(r) #r.gsub!(/\-/, '\-')
ret.scan(/\{#{r}[\&.*\;]*[\|[^\|^\}]*]*\}/).each { |s|
ret.gsub!(s, s.gsub(/\{#{r}[\&.*\;]*\|/, '{'))
#ret.gsub(s, s.gsub(/\{#{r}[\&.*\;]*\|/, '{'))
}
}
# apply spintax
ret = ret.spin
# return
return ret
end # def merged_subject
# If body is nil, return nil.
# raise an exception if track_clicks is activated and the outreach has not a profile assigned.
# raise an exception if track_opens is activated and the outreach has not a profile assigned.
# if the outreach has been created by an action, and the action.tracking_clicks in on, and the action.body_type is :html, then replace merge-tags in the body.
# if the outreach has been created by an action, and the action.tracking_opens in on, and the action.body_type is :html, then inject tracking pixel.
# Replace merge-tags.
# Replace still peding merge-tags by their fallback spintax.
# Apply spintax.
#
# Parameters:
# - preview: boolean. If true, don't apply the open-tracking pixel, and don't apply the tracking links.
#
def merged_body(preview: false)
return nil if self.body.nil?
ret = self.body.dup
track_clicks = false
track_opens = false
body_type = Mass::Action.body_code(:plain)
if self.action
track_clicks = self.action.track_clicks
track_opens = self.action.track_opens
body_type = self.action.body_type
end
err = []
# raise an exception if track_clicks is activated and the outreach has not a profile assigned.
err << "merged_body: track_clicks cannot be true if the outreach has not a profile assigned." if track_clicks && self.id_profile.nil?
# raise an exception if track_opens is activated and the outreach has not a profile assigned.
err << "merged_body: track_opens cannot be true if the outreach has not a profile assigned." if track_opens && self.id_profile.nil?
#
raise err.join(' ') if err.size > 0
# apply tracking links.
# iterate all href attributes of anchor tags
# reference: https://stackoverflow.com/questions/53766997/replacing-links-in-the-content-with-processed-links-in-rails-using-nokogiri
if self.action && self.action.track_clicks && !preview
n = 0
p = self.profile
fragment = Nokogiri::HTML.fragment(ret)
fragment.css("a[href]").each do |link|
# increment the URL counter
n += 1
# get the link
l = Mass::Link.where(:id_action=>self.action.id, :url=>link['href'], :delete_time=>nil).first
## replace the link with the tracking link
click_tracking_url = "#{p.tracking_url}/c/#{self.id}/#{l.id}"
link.content = click_tracking_url if link.text == link['href'] && self.action.body_type == Mass::Action.body_code(:plain)
link['href'] = click_tracking_url
end
# update notification body.
ret = fragment.to_html
# this forcing is because of this glitch: https://github.com/sparklemotion/nokogiri/issues/1127
ret.gsub!(/#{Regexp.escape('&')}/, '&')
# return
ret
end # if track_clicks
# apply tracking pixel
if self.action && self.action.track_opens && self.action.body_type == Mass::Action.body_code(:html) && !preview
p = self.profile
pixel = "<img src='#{p.tracking_url}/o/#{self.id}' height='1px' width='1px' />"
if ret =~ /<\/body>/
ret = ret.gsub(/<\/body>/, "#{pixel}</body>")
else
ret += pixel
end
end # if track_opens
# apply merge-tags, fallbacks and spintax
ret = self.generate_merged_text(ret, preview: preview)
# return
return ret
end # def merged_body
def merged_subject
return nil if self.subject.nil?
ret = self.subject.dup
ret = self.generate_merged_text(ret)
return ret
end # def merged_subject
# return a Sequel dataset, based on some filters.
# this method is used by the API to get the data from the database remotely
#
# Supported filters:
# - id_profile: string. Name of the profile type. Key sensitive.
# - status: string. Status of the outreach. Key sensitive. Allowed values are: pending, error, done.
# - approved: boolean. If the outreach is approved or not.
# - id_outreach_type: string. Name of the outreach type. Key sensitive.
# - id_tag: string. ID of the tag. Key sensitive.
# - direction: string. Direction of the outreach. Key sensitive. Allowed values are: incoming, outgoing, accepted.
# - basic: boolean. If the outreach is basic access or not.
#
def self.list(account, filters:)
err = key_errors(filters, allowed_keys: [:id_profile, :id_lead, :id_company, :status, :approved, :outreach_type, :outreach_type_desc, :tag, :direction, :basic, :reply_to_tag])
ds = self.base_list(account, filters: filters)
filters.each { |k, v|
if k.to_s == 'id_profile'
if !account
err << "Account is required to filter by id_profile."
else
p = Mass::Profile.where(:id => v.to_s, :id_account => account.id).first
if p.nil?
err << "Unknown profile #{v.to_s}."
else
ds = ds.where(:id_profile => p.id)
end
end
elsif k.to_s == 'id_lead'
if !account
err << "Account is required to filter by id_lead."
else
p = Mass::Lead.where(:id => v.to_s, :id_account => account.id, :delete_time => nil).first
if p.nil?
err << "Unknown lead #{v.to_s}."
else
ds = ds.where(:id_lead => p.id)
end
end
elsif k.to_s == 'id_company'
if !account
err << "Account is required to filter by id_company."
else
p = Mass::Company.where(:id => v.to_s, :id_account => account.id, :delete_time => nil).first
if p.nil?
err << "Unknown company #{v.to_s}."
else
ds = ds.where(:id_company => p.id)
end
end
elsif k.to_s == 'status'
code = self.status_code(v.to_sym)
err << "Unknown status #{v.to_s}. Allowed values are: #{statuses.join(', ')}" if code.nil?
ds = ds.where(:status => code) if code
elsif k.to_s == 'approved'
ds = ds.where(:approved => v.to_s)
elsif k.to_s == 'outreach_type'
if !account
err << "Account is required to filter by outreach_type."
else
ot = Mass::OutreachType.where(:id_account => account.id, :name => v.to_s).first
err << "Unknown outreach type #{v.to_s}." if ot.nil?
ds = ds.where(:id_outreach_type => ot.id) if ot
end
elsif k.to_s == 'tag'
if !account
err << "Account is required to filter by tag."
else
tag = Mass::Tag.where(:id_account => account.id, :name => v.to_s).first
err << "Unknown tag #{v.to_s}." if tag.nil?
ds = ds.where(:id_tag => tag.id) if tag
end
# only :outgoing outreaches has tags.
# both :incoming and :accepted outreaches have not tag assigned.
# if I am looking for :incoming or :accepted who with a previous :outgoing tag, then I must use the `reply_to_tag` filter.
elsif k.to_s == 'reply_to_tag'
if !account
err << "Account is required to filter by reply_to_tag."
else
tag = Mass::Tag.where(:id_account => account.id, :name => v.to_s).first
# get the first outreach with :outgoing directiom, with done_time not null, and with the tag; sorted by done_time desc.
ds = ds.where(
Sequel.lit("
(
select o2.id_tag
from outreach o2
where
o2.id_account = outreach.id_account AND
coalesce(o2.id_profile,'897b4c5e-692e-400f-bc97-8ee0e3e1f1cf') = coalesce(outreach.id_profile,'897b4c5e-692e-400f-bc97-8ee0e3e1f1cf') AND
coalesce(o2.id_lead,'897b4c5e-692e-400f-bc97-8ee0e3e1f1cf') = coalesce(outreach.id_lead,'897b4c5e-692e-400f-bc97-8ee0e3e1f1cf') AND
coalesce(o2.id_company,'897b4c5e-692e-400f-bc97-8ee0e3e1f1cf') = coalesce(outreach.id_company,'897b4c5e-692e-400f-bc97-8ee0e3e1f1cf') AND
o2.id_outreach_type = outreach.id_outreach_type AND
--o2.id_tag = '#{tag.id}' AND
o2.status = #{self.status_code(:performed)} AND
o2.direction = #{self.direction_code(:outgoing)} AND
o2.delete_time IS NULL AND
o2.done_time IS NOT NULL AND
o2.done_time < outreach.create_time
order by o2.done_time desc
limit 1
) = '#{tag.id}'
")
)
end
elsif k.to_s == 'direction'
ds = ds.where(:direction => self.direction_code(v.to_sym))
elsif k.to_s == 'basic'
# get list of all enrichment_types with basic access
outreach_types = []
if account
outreach_types = Mass::OutreachType.where(:id_account => account.id, :delete_time => nil).all.select { |o| o.basic_access? }
else
outreach_types = Mass::OutreachType.where(:delete_time => nil).all.select { |o| o.basic_access? }
end
ds = ds.where(:id_outreach_type => outreach_types.map { |o| o.id }) if v
ds = ds.exclude(:id_outreach_type => outreach_types.map { |o| o.id }) if !v
else
err << "Unknown filter #{k.to_s} for #{self.name}.list method."
end
}
raise err.join("\n") if err.size > 0
return ds
end # def self.list
# return an array of error messages
def self.errors(h={})
ret = []
ret += self.key_errors(h, allowed_keys: [
:id_account,
:id_user,
:profile,
:tag,
:approved,
:skip_repliers,
:direction,
:status, :outreach_type, :lead, :company, :subject, :body, :body_type, :error_description, :screenshot_url, :snapshot_url,
:message_id, :reply_to_message_id,
# dummy parameters, not stored in the database by returned by to_h method
:outreach_type_desc,
:merged_subject, :merged_body, :merged_body_text, :lead_or_company,
:action,
])
ret += self.ownership_errors(h)
ret += self.mandatory_errors(h, keys: [
:outreach_type, :status
])
ret += self.boolean_errors(h, keys: [:approved])
ret += self.url_errors(h, keys: [ :screenshot_url ])
ret += self.string_errors(h, keys: [ :subject, :body, :error_description, :message_id, :reply_to_message_id])
ret << "Value `id` must be a GUID." if h['id'] && !h['id'].to_s.guid?
ret << "Value `status` must be a symbol or string." if h['status'] && !h['status'].is_a?(Symbol) && !h['status'].is_a?(String)
ret << "Unkown `status` #{h['status']}. Allowed values are: #{statuses.join(', ')}" if h['status'] && !statuses.include?(h['status'].to_sym)
ret << "Unknown skip repliers `#{h['skip_repliers'].to_s}`." if h['skip_repliers'] && !h['skip_repliers'].nil? && !skips.include?(h['skip_repliers'].to_sym)
ret << "Value `direction` must be a symbol or string." if h['direction'] && !h['direction'].is_a?(Symbol) && !h['status'].is_a?(String)
ret << "Unkown `direction` #{h['direction']}. Allowed values are: #{directions.join(', ')}" if h['direction'] && !directions.include?(h['direction'].to_sym)
ret << "Value `body_type` must be a symbol or string." if h['body_type'] && !h['body_type'].is_a?(Symbol) && !h['body_type'].is_a?(String)
ret << "Unkown `body_type` #{h['body_type']}. Allowed values are: #{bodies.join(', ')}" if h['body_type'] && !bodies.include?(h['body_type'].to_sym)
ret << "Value `outreach_type` must be a symbol or string." if h['outreach_type'] && !h['outreach_type'].is_a?(Symbol) && !h['outreach_type'].is_a?(String)
ret << "Unkown `outreach_type` #{h['outreach_type']}." if h['outreach_type'] && !Mass::OutreachType.where(:id_account => h['id_account'], :name => h['outreach_type'].to_s, :delete_time => nil).first
ret << "Value `tag` must be a symbol or string." if h['tag'] && !h['tag'].is_a?(Symbol) && !h['tag'].is_a?(String)
ret << "Unkown `tag` #{h['tag']}." if h['tag'] && !Mass::Tag.where(:id_account => h['id_account'], :name => h['tag'].to_s).first
ret << "Value `subject` must be a string." if h['subject'] && !h['subject'].is_a?(String)
ret << "Value `body` must be a string." if h['body'] && !h['body'].is_a?(String)
ret << "Value `error_description` must be a string." if h['error_description'] && !h['error_description'].is_a?(String)
# message_id is allowd if the profile_type of the outreach is :mta only
if h['message_id'] || h['reply_to_message_id']
ot = Mass::OutreachType.where(:id_account => h['id_account'], :name => h['outreach_type'].to_s, :delete_time => nil).first
ret << "message_id is allowed if the outreach_type is :mta only." if h['message_id'] && ot && ot.profile_type.access != Mass::ProfileType.access_code(:mta)
ret << "reply_to_message_id is allowed if the outreach_type is :mta only." if h['reply_to_message_id'] && ot && ot.profile_type.access != Mass::ProfileType.access_code(:mta)
end
# either lead or company or lead_or_company must be present
#ret << "Either `lead` or `company` must be present." if !h['lead'] && !h['company']
ret << "Either `lead` or `company` or `lead_or_company` must be present." if !h['lead'] && !h['company'] && !h['lead_or_company']
# only one of lead or company or lead_or_company must be present
#ret << "Both `lead` and `company` cannot be present at the same time." if h['lead'] && h['company']
ret << "Only one of `lead` or `company` or `lead_or_company` must be present." if h['lead'] && h['company']
ret << "Only one of `lead` or `company` or `lead_or_company` must be present." if h['lead'] && h['lead_or_company']
ret << "Only one of `lead` or `company` or `lead_or_company` must be present." if h['company'] && h['lead_or_company']
#if h['outreach_type']
# i = h['outreach_type']
# i['id_account'] = h['id_account']
# i['id_user'] = h['id_user']
# ret += Mass::OutreachType::errors(i)
#end
if h['lead']
i = h['lead']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
ret += Mass::Lead::errors(i)
end
if h['company']
i = h['company']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
ret += Mass::Company::errors(i)
end
if h['lead_or_company']
i = h['lead_or_company']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
ret1 = Mass::Lead::errors(i)
ret2 = Mass::Company::errors(i)
if ret1.size > 0 && ret2.size > 0
ret << "The `lead_or_company` is not a valid lead and is not a valid company."
ret += ret1
ret += ret2
end
end
return ret
end # def errors
# Find an outreach by id.
# Find an outreach by id_profile, id_lead and body.
# Find an outreach by id_profile, id_company and body.
def self.find(h={})
o = nil
l = nil
if h['lead']
h['lead']['id_account'] = h['id_account']
h['lead']['id_user'] = h['id_user']
l = Mass::Lead.find(h['lead'])
end
c = nil
if h['company']
h['company']['id_account'] = h['id_account']
h['company']['id_user'] = h['id_user']
c = Mass::Company.find(h['company'])
end
p = nil
if h['profile']
h['profile']['id_account'] = h['id_account']
h['profile']['id_user'] = h['id_user']
p = Mass::Profile.find(h['profile'])
end
if o.nil? && h['id']
o = self.where(:id_account => h['id_account'], :id => h['id']).first
end
if o.nil? && p && h['lead']
o = self.where(:id_account => h['id_account'], :id_profile => p.id, :id_lead => l.id, :body => h['body']).first if p && l
end
if o.nil? && p && h['company']
c = Mass::Company.find(h['company'])
o = self.where(:id_account => h['id_account'], :id_profile => p.id, :id_company => c.id, :body => h['body']).first if p && c
end
return o
end # def self.find
# monkey patching the upsert method defined in the module InsertUpdate.
#
# If the profile assigned to this outreach is leased, then:
# - if the lead or doesn't exist in the CRM of the accunt, ignore the outreach.
# Else:
# - Upsert the outreach with its leads and company as is.
#
# If the lead has repied from a differnt email address, then:
# - Find the lead by finding the previous outreach with the `reply_to_message_id` equals to its `message_id`.
# - Save this new email in the email_2 field of the lead.
#
# parameters:
# - upsert_children: call upsert on children objects, or call insert on children objects
#
# return the object
def self.upsert(h={}, upsert_children: true)
el = nil
ec = nil
if h['profile']['leased']
if h['lead']
i = h['lead']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
el = Mass::Lead.find(i)
# if I didn't find the lead, try to find it using the email_2 field.
if el.nil?
el = Mass::Lead.where(
Sequel.lit("
id_account = '#{i['id_account']}' AND
delete_time IS NULL AND
email_2 IS NOT NULL AND
email_2 = '#{i['email'].to_s.to_sql}'
")
).first
end # el.nil?
elsif h['company']
i = h['company']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
ec = Mass::Company.find(i)
# if I didn't find the lead, try to find it using the email_2 field.
if ec.nil?
ec = Mass::Company.where(
Sequel.lit("
id_account = '#{i['id_account']}' AND
delete_time IS NULL AND
email_2 IS NOT NULL AND
email_2 = '#{i['email'].to_s.to_sql}'
")
).first
end # ec.nil?
elsif h['lead_or_company']
i = h['lead_or_company']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
ret1 = Mass::Lead::errors(i)
ret2 = Mass::Company::errors(i)
buffl = nil
buffc = nil
if ret1.size == 0
buffl = Mass::Lead.where(:id_account=>i['id_account'], :delete_time=>nil, :email=>i['email']).first
# if I didn't find the lead, try to find it using the email_2 field.
if buffl.nil?
buffl = Mass::Lead.where(
Sequel.lit("
id_account = '#{i['id_account']}' AND
delete_time IS NULL AND
email_2 IS NOT NULL AND
email_2 = '#{i['email'].to_s.to_sql}'
")
).first
end # buffl.nil?
end
if ret2.size == 0 && buffl.nil?
buffc = Mass::Company.where(:id_account=>i['id_account'], :delete_time=>nil, :email=>i['email']).first
# if I didn't find the lead, try to find it using the email_2 field.
if buffc.nil?
buffc = Mass::Company.where(
Sequel.lit("
id_account = '#{i['id_account']}' AND
delete_time IS NULL AND
email_2 IS NOT NULL AND
email_2 = '#{i['email'].to_s.to_sql}'
")
).first
end # buffc.nil?
end
if buffl
el = buffl
elsif buffc
ec = buffc
# by default, insert a new lead
else
el = Mass::Lead.find(i)
# if I didn't find the lead, try to find it using the email_2 field.
if el.nil?
el = Mass::Lead.where(
Sequel.lit("
id_account = '#{i['id_account']}' AND
delete_time IS NULL AND
email_2 IS NOT NULL AND
email_2 = '#{i['email'].to_s.to_sql}'
")
).first
end # el.nil?
end
end # if h['lead'] || h['company'] || h['lead_or_company']
end # if h['profile']['leased']
# if direction is :incoming or :accept, find another outreach where its
# message_id matches with the reply_to_message_id if this, and copy the lead and company.
if el.nil? && ec.nil?
if h['direction'].to_sym == :incoming || h['direction'].to_sym == :accepted
if h['reply_to_message_id']
o2 = self.where(:id_account => h['id_account'], :message_id => h['reply_to_message_id']).first
if o2
if o2.lead
el = o2.lead
el.email_2 = h['lead']['email'] if h['lead'] && h['lead']['email']
el.email_2 = h['lead_or_company']['email'] if h['lead_or_company'] && h['lead_or_company']['email']
el.save
elsif o2.company
ec = o2.company
ec.email_2 = h['company']['email'] if h['company'] && h['company']['email']
ec.email_2 = h['lead_or_company']['email'] if h['lead_or_company'] && h['lead_or_company']['email']
ec.save
end
end
end # if h['reply_to_message_id']
end # if h['direction'] == :incoming || h['direction'] == :accepted
end # if e.nil?
# only register the outreach if the lead or company exists in the CRM, or if the profile is not leased (so the account can ingest all the inbox)
if el || ec || !h['profile']['leased']
h['lead'] ||= el.to_h if el
h['company'] ||= ec.to_h if ec
h['lead_or_company'] = nil if el || ec
o = self.find(h)
b = upsert_children
if o.nil?
o = self.insert(h, upsert_children: b)
else
o.update(h, upsert_children: b)
end
return o
else
return nil
end
end # def self.upsert
# insert or update a record
# return the object
def update(h={}, upsert_children: true)
b = upsert_children
o = base_update(h, upsert_children: b, manage_done_time: true)
o.status = self.class.status_code(h['status']) if h['status']
o.direction = self.class.direction_code(h['direction']) if h['direction']
o.approved = h['approved'] if h['approved']
o.skip_repliers = self.class.skip_code(h['skip_repliers']) if !h['skip_repliers'].nil?
o.id_tag = Mass::Tag.where(:id_account => h['id_account'], :name => h['tag'].to_s).first.id if h['tag']
o.subject = h['subject'] if h['subject']
o.body = h['body'] if h['body']
o.body_type = self.class.body_code(h['body_type']) if h['body_type']
o.error_description = h['error_description'] if h['error_description']
o.screenshot_url = h['screenshot_url'] if !h['screenshot_url'].nil?
o.snapshot_url = h['snapshot_url'] if !h['snapshot_url'].nil?
o.message_id = h['message_id'] if h['message_id']
o.reply_to_message_id = h['reply_to_message_id'] if h['reply_to_message_id']
o.id_outreach_type = Mass::OutreachType.where(:id_account => h['id_account'], :name => h['outreach_type'].to_s).first.id if h['outreach_type']
#if h['outreach_type']
# i = h['outreach_type']
# i['id_account'] = h['id_account']
# i['id_user'] = h['id_user']
# e = Mass::OutreachType.upsert(i)
# o.id_outreach_type = e.id
#end
#
if h['lead']
i = h['lead']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
e = Mass::Lead.upsert(i)
o.id_lead = e.id
elsif h['company']
i = h['company']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
e = Mass::Company.upsert(i)
o.id_company = e.id
elsif h['lead_or_company']
i = h['lead_or_company']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
ret1 = Mass::Lead::errors(i)
ret2 = Mass::Company::errors(i)
buffl = nil
buffc = nil
if ret1.size == 0
buffl = Mass::Lead.where(:id_account=>self.id_account, :delete_time=>nil, :email=>i['email']).first
end
if ret2.size == 0 && buffl.nil?
buffc = Mass::Company.where(:id_account=>self.id_account, :delete_time=>nil, :email=>i['email']).first
end
if buffl
buffl.update(i)
o.id_lead = buffl.id
elsif buffc
e = buffc.update(i)
o.id_company = buffc.id
# by default, insert a new lead
else
e = Mass::Lead.upsert(i)
o.id_lead = e.id
end
end
if h['profile']
i = h['profile']
i['id_account'] = h['id_account']
i['id_user'] = h['id_user']
e = Mass::Profile.upsert(i)
o.id_profile = e.id
end
o.save
return o
end # def update
def to_h(preview: false)
ret = self.to_h_base
ret['status'] = self.class.statuses[self.status] if !self.status.nil?
ret['tag'] = self.tag.name.to_s if !self.tag.nil?
ret['approved'] = self.approved if !self.approved.nil?
ret['skip_repliers'] = self.class.skips[self.skip_repliers] if !self.skip_repliers.nil?
ret['direction'] = self.class.directions[self.direction] if !self.direction.nil?
ret['error_description'] = self.error_description if self.error_description
ret['screenshot_url'] = self.screenshot_url if !self.screenshot_url.nil?
ret['snapshot_url'] = self.snapshot_url if !self.snapshot_url.nil?
ret['outreach_type'] = self.outreach_type.name.to_sym if !self.outreach_type.nil?
ret['outreach_type_desc'] = self.outreach_type.to_h if !self.outreach_type.nil?
ret['lead'] = self.lead.to_h if !self.lead.nil?
ret['company'] = self.company.to_h if !self.company.nil?
ret['profile'] = self.profile.to_h if !self.profile.nil?
ret['subject'] = self.subject if !self.subject.nil?
ret['body'] = self.body if !self.body.nil?
ret['body_type'] = self.class.bodies[self.body_type] if !self.body_type.nil?
ret['message_id'] = self.message_id if !self.message_id.nil?
ret['reply_to_message_id'] = self.reply_to_message_id if !self.reply_to_message_id.nil?
# dummy parameters, not stored in the database by returned by to_h method
ret['merged_subject'] = self.merged_subject
ret['merged_body'] = self.merged_body(preview: preview)
doc = Nokogiri::HTML(self.merged_body(preview: preview))
doc.search('a').each{ |a|
if a.text == a['href']
a.replace "#{a['href']}"
else
a.replace "#{a.text}: #{a['href']}"
end
}
doc.search('p,div,br').each{ |e| e.after "\n" }
ret['merged_body_text'] = doc.text
ret['action'] = self.action.nil? ? nil : self.action.to_h
# return
return ret
end # def to_h
# iterate pendings outreaches, with no profile assigned, not deleted and belonging to an active outreach
def self.assign(logger:nil)
l = logger || BlackStack::DummyLogger.new(nil)
l.log "Starting outreach assignation.".blue
q = "
SELECT e.id
FROM \"outreach\" e
-- job cannot be assigned to a basic-access profile
JOIN \"outreach_type\" et ON e.id_outreach_type=et.id
JOIN \"profile_type\" pt ON (et.id_profile_type=pt.id AND pt.access<>#{Mass::ProfileType.access_code(:basic)})
-- job must be pending, approved, with no profile assigned and not deleted
WHERE e.status=#{status_code(:pending)}
AND e.direction=#{direction_code(:outgoing)}
AND e.approved = true
AND e.id_profile IS NULL
AND e.delete_time IS NULL
ORDER BY e.create_time
"
DB[q].all { |row|
l.logs "Assigning outreach #{row[:id].blue}... "
Mass::Outreach.where(:id => row[:id]).first.assign(logger: l)
l.logf 'done'.green
}
end # def self.assign
# iterate profiles:
# - belonging to the same account,
# - not deleted,
# - not in :idle state,
# - belonging to the same tag than the outreach,
# - belonging to the same profile_type than the outreach.
#
# Get the list of profiles of this account, belonging to the same channel than the outreach.
#
# If the lead/company has been already reached by another outreach with the same profile_type,
# and the profile of that previous outreach is not deleted, and it is still belonging to this account,
# then this is the only profile that can be assigned to this outreach.
#
# if the profile is assigned to any pending outreach, skip it.
# if the profile is assigned to a outreach with a done_time within the last outreaches_interval of the profile, skip it.
# if the number of outreaches with done_time within the last day and status == :performed or :failed or :running, is higher than the max_daily_processed_outreaches of the profile, skip it.
# if the number of outreaches with done_time within the last day and status == :aborted is higher than the max_daily_aborted_outreaches of the profile, skip it.
# else, assign the profile to the job and exit the look.
#
def assign(logger:nil)
l = logger || BlackStack::DummyLogger.new(nil)
support_state = self.outreach_type.profile_type.rpa?
# If the lead/company has been already reached by another outreach with the same profile_type,
# and the profile of that previous outreach is not deleted, and it is still belonging to this account,
# then this is the only profile that can be assigned to this outreach.
pt = self.outreach_type.profile_type
prev_profile = nil
if self.lead
q = "
select o.id, o.id_profile, o.status
from outreach o
join outreach_type ot on ot.id=o.id_outreach_type
where ot.id_profile_type = '#{pt.id}'
and o.status = #{self.class.status_code(:performed)}
and o.direction = #{self.class.direction_code(:outgoing)}
and o.id_account = '#{self.id_account}'
"
q += "
and o.id_lead = '#{self.id_lead}'
" if self.id_lead
q += "
and o.id_company = '#{self.id_company}'
" if self.id_company
q += "
order by o.done_time desc
"
row = DB.fetch(q).first
if row
pid = row[:id_profile]
prev_profile = Mass::Profile.where(:id=>pid, :id_account=>self.id_account, :delete_time=>nil).first if pid
else
prev_profile = nil
end
end
# iterate profiles:
# - belonging to the same account,
# - not deleted,
# - not in :idle state,
# - belonging to the same tag than the outreach,
# - belonging to the same profile_type than the outreach_type of the outreach.
q = "
SELECT p.id
FROM \"profile\" p
JOIN \"account\" a ON (p.id_account=a.id AND a.id='#{self.id_account}')
WHERE p.delete_time IS NULL
#{support_state ? "AND p.state<>#{self.class.state_code(:idle)}" : ''}
AND p.id_profile_type = '#{self.outreach_type.id_profile_type}'
"
q += "
AND p.id = '#{prev_profile.id}'
" if prev_profile
q += "
AND p.id_tag = '#{self.id_tag}'
" if self.id_tag
DB[q].all { |row|
l.logs "Profile (#{row[:id].blue})... "
p = Mass::Profile.where(:id => row[:id]).first
# if the profile is assigned to any pending job, skip it.
n = Mass::Outreach.where(
:id_profile => p.id,
:status => self.class.status_code(:pending),
:direction => self.class.direction_code(:outgoing),
:approved => true
).count
if n > 0
l.logf 'skip'.yellow + ' (already assigned to pending Outreach)'
next
end
# if the profile is assigned to a outreach with a done_time within the last outreaches_interval of the profile, skip it.
n = Mass::Outreach.where(
Sequel.lit("
id_profile='#{p.id}' AND
status <> #{self.class.status_code(:pending)} AND
COALESCE(done_time, CAST('1900-01-01' AS TIMESTAMP)) > CAST('#{now}' AS TIMESTAMP) - INTERVAL '#{p.outreach_interval.to_s} seconds'
")
).count
if n > 0
l.logf 'skip'.yellow + " (already perfomed an outreach within the last #{p.outreach_interval.to_s} seconds)"
next
end
# if the number of outreaches with done_time within the last day and status == :performed or :failed or :running, is higher than the max_daily_processed_outreaches of the profile, skip it.
n = Mass::Outreach.where(
Sequel.lit("
id_profile='#{p.id}' AND
status IN (#{self.class.status_code(:running)}, #{self.class.status_code(:performed)}, #{self.class.status_code(:failed)}) AND
COALESCE(done_time, CAST('1900-01-01' AS TIMESTAMP)) > CAST('#{now}' AS TIMESTAMP) - INTERVAL '24 HOURS'
")
).count
if n > p.max_daily_processed_outreaches
l.logf 'skip'.yellow + ' (daily outreaches quota)'
next
end
# if the number of outreaches with done_time within the last day and status <> :pending is higher than the max_daily_processed_outreaches of the profile, skip it.
n = Mass::Outreach.where(
Sequel.lit("
id_profile='#{p.id}' AND
status <> #{self.class.status_code(:pending)} AND
status <> #{self.class.status_code(:aborted)} AND
COALESCE(done_time, CAST('1900-01-01' AS TIMESTAMP)) > CAST('#{now}' AS TIMESTAMP) - INTERVAL '24 HOURS'
")
).count
if n > p.max_daily_processed_outreaches
l.logf 'skip'.yellow + ' (daily processed outreaches quota)'
next
end
# if the number of outreaches with done_time within the last day and status == :aborted is higher than the max_daily_aborted_outreaches of the profile, skip it.
n = Mass::Outreach.where(
Sequel.lit("
id_profile='#{p.id}' AND
status = #{self.class.status_code(:aborted)} AND
COALESCE(done_time, CAST('1900-01-01' AS TIMESTAMP)) > CAST('#{now}' AS TIMESTAMP) - INTERVAL '24 HOURS'
")
).count
if n > p.max_daily_aborted_outreaches
l.logf 'skip'.yellow + ' (daily aborted outreaches quota)'
next
end
# else, assign the profile to the job
self.id_profile = p.id
self.save
l.logf 'done'.green
# and exit the look
break
}
end # def assign
# apply this outreach to the timeline
def timeline(logger:nil)
l = logger || BlackStack::DummyLogger.new(nil)
end # def timeline
# pass this outreach through all the active rules beloning to the same account
def rule(logger:nil)
l = logger || BlackStack::DummyLogger.new(nil)
e = this
Mass::Rule.where(
:id_account => e.id_account,
:active => true,
:delete_time => nil
).each { |r|
# iterate all the :event_created triggers belonging to the rule, linked to the same source than the event.
l.logs "Rule: #{r.name.blue}... "
Mass::Trigger.where(
:id_account => r.id_account,
:id_rule => r.id,
:trigger => [Mass::Trigger.trigger_code(:outreach_done), Mass::Trigger.trigger_code(:accepted_request)],
:id_outreach => e.id_outreach,
:delete_time => nil
).each { |t|
#
l.logs "Trigger: #{t.name.blue}... "
# pass the events, lead, company through the filters.
passed = true
Mass::Filter.where(
:id_account => r.id_account,
:id_rule => r.id,
:delete_time => nil
).each { |f|
l.logs "Filter: #{f.name.blue}... "
if f.type == Mass::Filter.filter_code(:lead) && e.lead
passed &&= f.test_lead(e.lead)
elsif f.type == Mass::Filter.filter_code(:company) && e.company
passed &&= f.test_company(e.company)
#elsif f.type == Mass::Filter.filter_code(:event) && e
# passed &&= f.test_event(e)
end
if passed
l.logs "passed".green
else
l.logs "failed".yellow
break
end
} # each filter
l.logf 'done'.green
if passed
# pass the event through the actions
Mass::Action.where(
:id_account => r.id_account,
:id_rule => r.id,
:delete_time => nil
).each { |a|
l.logs "Action: #{a.name.blue}... "
a.schedule(
delay_number: r.delay_number,
delay_unit: Mass::Rule.units[r.delay_unit],
lead: e.lead,
company: e.company #,
#tag: e.tag,
)
l.logf 'done'.green
} # each action
end
} # each trigger
l.logf 'done'.green
} # each rule
end # def rule
# get the latest and performed :outgoing outreach for the same profile and the same lead or company, sorting by the create_time field.
# raise an exception if the outreach has no lead and no company.
# raise an exception if no profile is assigned to this outreach.
def prev
id_profile = self.id_profile
id_lead = self.id_lead
id_company = self.id_company
raise "Cannot get previous outreach because no lead and no company are assigned to this outreach." if id_lead.nil? && id_company.nil?
raise "Cannot get previous outreach because no profile is assigned to this outreach." if id_profile.nil?
q = "
select o.id
from \"outreach\" o
where o.id_profile = '#{id_profile}'
and o.status = #{Mass::Outreach.status_code(:performed)}
and o.direction = #{Mass::Outreach.direction_code(:outgoing)}
and o.create_time < CAST('#{self.create_time}' AS TIMESTAMP)
"
q += "
and o.id_lead = '#{id_lead}'
" if id_lead
q += "
and o.id_company = '#{id_company}'
" if id_company
q += "
order by o.create_time desc
"
row = DB[q].first
return nil if row.nil?
return Mass::Outreach.where(:id => row[:id]).first
end
# timeline summarization methods
#
# - raise an exception if the direction of this is not :incoming or :accepted
# - (DEPRECATED) raise an exception if there is not a last outgoing outreach from the same
# profile to the same lead/company, and generated from an action.
#
# return true if:
#
# 1. there is not any previous outreach,
# 2. in the same direction between the same profile and lead/company,
# 3. after the last outgoing outreach from the same profile to the
# same lead/company, and generated from an action.
#
def first_time?
raise "Cannot check if it is the first time if the direction is not :incoming or :accepted." if self.direction != self.class.direction_code(:incoming) && self.direction != self.class.direction_code(:accepted)
# last outgoing outreach from the same profile to the same lead/company.
last_outgoing_outreach = Mass::Outreach.where(
:id_account => self.id_account,
:id_profile => self.id_profile,
:id_lead => self.id_lead,
:id_company => self.id_company,
:direction => self.class.direction_code(:outgoing),
:status => self.class.status_code(:performed),
).where(
Sequel.lit("
done_time is not null and
id_action is not null
")
).order(:done_time).last
# This validation is removed because of the following situation:
# 1. you add a lead to your CRM with who you already had a conversation in the past.
# 2. such a new conversation with the lead is scraped, and the first message in the conversation is :incoming.
#
#raise "The :incoming or :accepted outreach (#{self.id}) has not a previous :outgoing message generated by a rule." if last_outgoing_outreach.nil?
q = "
SELECT o.id
FROM \"outreach\" o
WHERE o.id_account = '#{self.id_account}'
AND o.direction = #{self.direction}
AND o.id_outreach_type = '#{self.id_outreach_type}'
AND o.id_profile = '#{self.id_profile}'
"
q += "
AND COALESCE(o.id_lead, 'b08d6d8b-5e0f-40a2-966c-673eb409ef34') = '#{self.id_lead}'
" if self.id_lead
q += "
AND COALESCE(o.id_company, 'b08d6d8b-5e0f-40a2-966c-673eb409ef34') = '#{self.id_company}'
" if self.id_company
q += "
AND o.create_time > CAST('#{last_outgoing_outreach.done_time}' AS TIMESTAMP)
" if last_outgoing_outreach
q += "
AND o.create_time < CAST('#{self.create_time}' AS TIMESTAMP)
"
DB[q].first.nil?
end
# if the direction is :outgoing,
# - load the action that created this outreach
# - find a record with the same id_outreach_type and id_action, and a null value in the field id_profile.
# - if the record doesn't exists, then create it.
# - increase the counter outreaches_created in such a record.
#
# update the flag summarize_creation_to_timeline of this object.
def summarize_creation_to_timeline
# if the direction is :outgoing,
if self.direction == self.class.direction_code(:outgoing)
# load the action that created this outreach
a = self.action
# find a record with the same id_outreach_type and id_action, and a null value in the field id_profile.
t = Mass::Timeline.where(
Sequel.lit("
id_outreach_type = '#{self.id_outreach_type}' AND
id_profile IS NULL AND
id_action = #{ a.nil? ? "NULL" : "'#{a.id}'" } AND
\"year\" = #{self.create_time.year} AND
\"month\" = #{self.create_time.month} AND
\"day\" = #{self.create_time.day} AND
\"hour\" = #{self.create_time.hour} AND
\"minute\" = #{self.create_time.min}
")
).first
# if the record doesn't exists, then create it.
if t.nil?
t = Mass::Timeline.new
t.id = guid
t.id_account = self.id_account
t.create_time = now
t.id_outreach_type = self.id_outreach_type
t.id_profile = nil
t.id_action = a.id if a
t.year = self.create_time.year
t.month = self.create_time.month
t.day = self.create_time.day
t.hour = self.create_time.hour
t.minute = self.create_time.min
t.dt = self.create_time
end
elsif self.direction == self.class.direction_code(:incoming) || self.direction == self.class.direction_code(:accepted)
# load previous :performed and :outgoing outreach
o = self.prev
# load action that created the previous outreach
a = o.nil? ? nil : o.action
# find a record with the same id_outreach_type and id_profile.
t = Mass::Timeline.where(
Sequel.lit("
id_outreach_type = '#{self.id_outreach_type}' AND
id_profile = '#{self.id_profile}' AND
id_action #{ a.nil? ? "IS NULL" : "= '#{a.id}'" } AND
\"year\" = #{self.create_time.year} AND
\"month\" = #{self.create_time.month} AND
\"day\" = #{self.create_time.day} AND
\"hour\" = #{self.create_time.hour} AND
\"minute\" = #{self.create_time.min}
")
).first
# if the record doesn't exists, then create it.
if t.nil?
t = Mass::Timeline.new
t.id = guid
t.id_account = self.id_account
t.create_time = now
t.id_outreach_type = self.id_outreach_type
t.id_profile = self.id_profile
t.id_action = a.id if a
t.year = self.create_time.year
t.month = self.create_time.month
t.day = self.create_time.day
t.hour = self.create_time.hour
t.minute = self.create_time.min
t.dt = self.create_time
end
end # if self.direction == self.class.direction_code(:outgoing)
if t
# if the direction is :outgoing, then increase the counter outreaches_created in the record.
if self.direction == self.class.direction_code(:outgoing)
t.outreaches_created = t.outreaches_created.to_i + 1
end # if self.direction == self.class.direction_code(:outgoing)
# if the direction is :incoming, then increase the counter outreaches_replied in the record.
if self.direction == self.class.direction_code(:incoming)
t.outreaches_replied = t.outreaches_replied.to_i + 1
t.outreaches_unique_replied = t.outreaches_unique_replied.to_i + 1 if first_time?
end # if self.direction == self.class.direction_code(:incoming)
# if the direction is :accepted, then increase the counter outreaches_accepted in the record.
if self.direction == self.class.direction_code(:accepted)
t.outreaches_accepted = t.outreaches_accepted.to_i + 1
t.outreaches_unique_accepted = t.outreaches_unique_accepted.to_i + 1 if first_time?
end # if self.direction == self.class.direction_code(:accepted)
t.save
end
# update the flag summarize_creation_to_timeline of this object.
self.summarize_creation_to_timeline = now
self.save
end # summarize_creation_to_timeline
# raise an exception of id_profile is null.
# raise an exception of done_time is null.
# load the action that created this outreach
# find a record with the same id_outreach_type and id_profile.
# if trhe record doesn't exists, then create it.
# if the direction is :outgoing and the status is :performed, then increase the counter outreaches_performed in the record.
# if the direction is :outgoing and the status is :failed, then increase the counter outreaches_failed in the record.
# if the direction is :outgoing and the status is :aborted, then increase the counter outreaches_aborted in the record.
# save the timeline record.
# update the flag summarize_status_to_timeline of this object.
def summarize_status_to_timeline
if self.direction == self.class.direction_code(:outgoing)
# raise an exception of id_profile is null.
#raise 'id_profile cannot be null' if self.id_profile.nil?
# raise an exception of done_time is null.
raise 'done_time cannot be null' if self.done_time.nil?
# load the action that created this outreach
a = self.action
# find a record with the same id_outreach_type and id_profile.
t = Mass::Timeline.where(
Sequel.lit("
id_outreach_type = '#{self.id_outreach_type}' AND
id_profile #{ id_profile.nil? ? "IS NULL" : "= '#{id_profile}'" } AND
id_action #{ a.nil? ? "IS NULL" : "= '#{a.id}'" } AND
\"year\" = #{self.done_time.year} AND
\"month\" = #{self.done_time.month} AND
\"day\" = #{self.done_time.day} AND
\"hour\" = #{self.done_time.hour} AND
\"minute\" = #{self.done_time.min}
")
).first
# if the record doesn't exists, then create it.
if t.nil?
t = Mass::Timeline.new
t.id = guid
t.id_account = self.id_account
t.create_time = now
t.id_outreach_type = self.id_outreach_type
t.id_profile = self.id_profile
t.id_action = a.id if a
t.year = self.done_time.year
t.month = self.done_time.month
t.day = self.done_time.day
t.hour = self.done_time.hour
t.minute = self.done_time.min
t.dt = self.done_time
end
# if the status is :performed, then increase the counter outreaches_performed in the record.
t.outreaches_performed = t.outreaches_performed.to_i + 1 if self.status == self.class.status_code(:performed)
# if the status is :failed, then increase the counter outreaches_failed in the record.
t.outreaches_failed = t.outreaches_failed.to_i + 1 if self.status == self.class.status_code(:failed)
# if the status is :aborted, then increase the counter outreaches_aborted in the record.
t.outreaches_aborted = t.outreaches_aborted.to_i + 1 if self.status == self.class.status_code(:aborted)
# and save the timeline record.
t.save
end
# update the flag summarize_status_to_timeline of this object.
self.summarize_status_to_timeline = now
self.save
end # def summarize_status_to_timeline
end # class Outreach
end # module Mass
Please privide the following Ruby code snippets to add to mass-copilot.rb:
a hash descriptor of a function create_outreach to add to the openai_client.
Use GPT for writing documentation.
Training Prompt 1
A skeleton class is a class developed in Ruby that inherits from Ruby Sequel and has direct access to the database. Instances of skeleton classes work at the server side.
A stub class is a class with API communication with the server. Instances of stub classes work at the client side. Instances of stub classes have a counter-part at the server side who generate the API responses.
Any stub class inherits directly or indirectly from BlackStack::Base:
Skeleton classes use to extend these Ruby modules:
Training Prompt 2
The communication between a stub instances and its counter-part skeleton instance is trough the `Mass::Client** library.
mass-client.rb:
Training Prompt 3
I am writing a copilot to operate my SaaS:
mass-client.rb:
Requirement Prompt Example
This is the stub class
Mass::OutreachThis is the skeleton class
Mass::OutreachPlease privide the following Ruby code snippets to add to
mass-copilot.rb:create_outreachto add to the openai_client.create_outreach