Flow¶
The flows are designed to build business rules and simplify the selection process of information, and to provide a visual picture of what information is flowing.
Advantages of Building Flows:
- Visual image of a flow/process.
- Simplifies the selection process (smaller selection)
- Combine selection from several different data sources.
- Ability to easily divide target groups into one or more legs.
- Ability to schedule / execute one or more parts of a flow.
- Ability to run one or more actions in one and the same flow.
- Build more advanced business rules.
How to create a new flow¶
1. Start by building up your structure with folders.
2. Select the folder to which you want to create a new flow for.
3. Click on the three-point menu to the right and select “New flow”
A blank page opens. To the right you can see the settings for the flow. Start by giving the flow a name.
Flow settings¶
Header¶
There are some functions in the head of the flow settings.
Active
“Active” is a checkbox that determines whether the flow is active or not. If a flow is not active, it cannot be run. Not even individual actions in the flow.
Export
“Export” is a function for exporting an entire flow structure so that it can then be imported into another environment. Exports carry the entire chain of actions. However, it does not take with you certain settings that can be made in an action because they can be tightly connected to the environment the action is configured for.
Delete
“Delete” deletes the entire flow. However, you cannot delete a feed containing participants. Before removing a flow, you must first empty the flow of participants.
Empty
“Empty” is a function for emptying your flow of participants.
More button
Button for displaying advanced flow settings. See Advanced settings
Identification for a flow¶
When creating a new flow, each flow gets its unique “Id”.
When you are on a flow you can also see its “Id” up in the browser’s URL.
By copying the url one can send it to other users as a direct link to the flow. Or if you want to navigate to the flow from other systems.
Types of flows¶
In Sweet Automation you can create two types of flows, “Standard” or “ETL”. When creating a new flow, do you have to decide what type of flow you want? “Sandard” is default, and the most common type of flows.
Flows can be used in several ways. Process target groups, automate processes, build integrations, etc.
Standard
A standard flow has actions and boxes. The flow is started from the top where in most cases a target group is loaded. The target group is in so-called boxes. Between each box there is an action that distributes or performs things on the target group. The target group is distributed through the flow. A flow can be run manually, via scheduling or triggered from outside.
FlowParticipants:
A flow can be loaded with anything
- Persons
- Companies
- Cases
- Activities
- Machines
- Products
- and so on
We call these target group for Participants or FlowParticipants.
Basic rule for standard flow:
A flow is loaded with a predefined key, FlowKey. This key should be unique to each participant to be loaded into the flow. (This is done in “Flow Settings”).
A flow is only loaded with unique keys (FlowKey) No duplicates are allowed.
When a participant gets loaded to a flow, it cannot re-enter. You can load as many times as you like, but the participants who are already loaded will not return (no duplicates).
The only way for a participant to re-enter a flow is:
- When the participant is deleted from the flow. Then its allowed to enter again.
- When the participant is disabled in the flow. Then its allowed to enter again. The disabled one stays where it is.
For each box you can see down to the right how many participants are in the box. The box can be opened to watch its participants.
ETL
In an ETL flow there is no information flowing. Instead, one can build a chain of actions that perform things. It may be, for example, to build integrations, move, update, create, export data for one or more data sources.
Preview Mode¶
“Preview mode” is default selected for all new flows. This means that no “dangerous” actions will run as long as the flow is in Preview mode. This is a security feature that allows you to be able to build and test your flows without risking, for example, sending mail or creating activities in other systems while building and testing. For each action in the system, one defines whether the action is dangerous or not.
Logging¶
“Logging” is not default when creating a new flow. If “Logging” is selected, all participants in a flow will log the way it flows through, when? where? how? This is saved in a separate table that is linked to your flow. Logging can generate very large amounts of data so you have to think about it so that the disk space does not run out or the space starts to cost a lot of unnecessary money.
Contains sensitive data¶
“Contains sensitive data” is merely a label if the flow contains sensitive data. This tag has a link to the “GDPR” action included with the system (The GDPR action is an action used to clear sensitive data in flows). In the GDPR action, you have the ability to filter out the flows that have just “Contain sensitive data” checked. The person who builds the flow must set this flag himself if the flow can contain sensitive data. Sensitive data can, for example, be personal data such as e-mail addresses, social security numbers or the like
Flow storage¶
“Flow storage” is a setting where you determine where your flow data should be stored. By default, the “Legacy” type is selected. With “Legacy” it means using the built-in structure that comes with Sweet Automation. Here, all flows are saved down in the same structure, ie a common storage area. If you choose to store your flow data elsewhere, you first need to create a connection in the admin tool. See below:
Legacy storage
“Legacy storage” saves all flow data into a common structure that comes with Sweet Automation. Driving in Legacy is very easy and easy to get started. However, the downside may be that over time, this structure can become very large and take up a lot of space. It can also affect performance. All data stored in the flow is also stored in a dynamic structure which is more difficult to use if you want to make selection directly against the flow.
Self storage
With “Self storage” you choose where your flow and the information in the flow should be saved. In order to make this possible, you first need to add a link in the administration tool. The link determines where you can save your flows. Today you can choose between:
- Microsoft SQL-Server
- My SQL
- Oracle
- Google Big Query
The connection can be both internal and external.
When in Flow Settings you choose your own storage location, a structure per flow will be created in this data source. This means that each flow has its own isolated structure. The structure is flat and built dynamically in conjunction with adding more data through actions. There is also created a separate selection data source for the flow, which allows you to easily build selection against a flow.
Base template¶
For a flow, you can define that a selection template should be used to return data to a flow box. Here you also determine which field / attribute to use as a flow key. Here you prefer to choose a field that is unique to the participants on which the flow is to be based. A flow can be based on anything, example:
- Persons (Flow key: PersonId, SSN, e-mail etc (something unique)).
- Companies (Flow key: CompanyId, OrganizationalNumber CustomerNumber etc (something unique)).
- Cases (Flow key: CaseId, CaseNumber (something unique)).
- Forms (Flow key: ResultSetId, ResultNumber (something unique)).
- Machines (Flow key: MachineId, RegistrationNumber, SerialNumber (something unique)).
- Product (Flow key: ProductId, ProductNumber, SerialNumber (something unique)).
- Bookings (Flow key: BookingId, BookingNumber (something unique)).
- Activities (Flow key: ActivityId (something unique)).
- etc
Selection template
To select a selection template can be done in two ways:
- Search the template via the drop down box, “Selection template”.
- Via drag and drop. Select selection template from your navigation tree and drop it in the “Selection Template” block.
In this case, I choose to drag in the template we created during the selection section, “All Persons”.
Selected template.
FlowKey
A “FlowKey” should be selected from the result columns included in the selected selection template. Remember! A FlowKey should be something unique from your selection template. In this example, we select “PersonId”.
Datatype
When FlowKey is selected, the data type for the field will automatically be set. If it is not successful or you want to choose another data type, you can choose another. The data types that can be selected are:
- Int Numerical value.
- String Text value.
- Uniqueidentifier GUID.
Our Flowkey (PersonId) is set to Unique identifier.
No Base template
If you do not choose to set a “Base template”, a flow box will display the data associated with your FlowKey. You do not need to concern anything in this in the first position. Some actions will itself put the information here if it is missing.
Locked level and visibility¶
This works the same way as for selection
Locked level
- Open = Selection is open for changes. Anyone can change this selection.
- Locked = Selection is locked for everyone except you and administrators.
- Sealed = Selection is locked for everyone, including you and administrators. It can only be unlocked with help from Sweet Systems.
Visible
- Checked = Selection is visible for everyone.
- Unckecked = Selection is only visible for you and administrators.
Advanced settings¶
When you open up more settings on the flows, the dialog is expanded and more fields become visible as below.
Start/End - Period
Start and end are date fields that are linked to the active period of the flow. Default is these empty which means that the flow has no active or inactive period. Here, only the “Active” flag determines whether the flow is active or not.
However, should one specify a period through Start and End then the flow will only be active within that period, provided the “Active” flag is checked. Outside this period, the flow will not be active, even if the “Active” flag is checked.
Key
“Key” is a free text field where you can give the flow a unique name that should not be visible in the interface. This key can then be used via the REST-Api that comes with Sweet Automation. The key can therefore be good if you want to call the flow.
Status
“Status” is metadata about a flow. It can be used to set the status of a flow. No functions are associated with status.
Category
“Category” is the metadata about the flow for how you would like to categorize the flow. This can be good for follow-up purposes if you have reports that should aggregate or group information about flows.
Purpose
“Purpose” is the metadata of the flow for how you would like to categorize the flow. This can be good for follow-up purposes if you have reports that should aggregate or group information about flows.
Budget
“Budget” is the metadata about the flow that can be used for follow-up to be able to calculate the outcomes for a flow
Expected ROI
“Expected ROI” is the metadata about the flow that can be used for follow-up to be able to calculate the outputs of a flow
Description
“Description” can be used to briefly describe what a flow does.
Flow setting done
When you’re done with your flow settings. Close the dialog via the close button at the top left of the dialog
Toolbox¶
To open the toolbox, click on the “Edit” button at the top right. The toolbox contains all the tools for building a flow or process. The toolbox is grouped into different sections for what type of tool it is. The content of the toolbox depends on the type of flow you are in. Standard flow or ETL flow.
Toolbox Standard flow¶
The content of the toolbox for standard flows contains actions that have been taken to work on standard flows. This toolbox is by default grouped in 5 parts Nodes, Loading Tasks, Move, Task and E-Marketing.
Nodes
Box can be pulled out and released on an action if the action allows it. Boxes are where participants are located.
Loading Tasks
DB is the most common charge action. It is a charge action that is based on a sample template. See DB fore more information.
Excel is an action to load a flow of information that is on file. Possible formats, xlsx, csv, txt. See File fore more information.
POP3 is an action that loads the feeds from an incoming mailbox. See POP3 fore more information.
Empty Is an empty action that cannot load anything. It is used in feeds born with data from elsewhere, via api or other flows. See Empty fore more information.
WS is an action where you can predefine the information that should be in the flow. The flow is then loaded via api calls. See WS fore more information.
Azure storage is an action to download information stored in an “Azure storage”. See Azure storage fore more information.
Move
Split is the most common action. It is used to build rules for distributing participants from one box to one or more. See Split fore more information.
Move is an action for moving participants from one box to another. Either within the same flow or another flow. See Move fore more information.
Copy is an action to copy participants to another feed. This cannot be used to copy within the same flow because a participant may only be present at one and the same flow. See Copy fore more information.
Trigger is an action that is used to trigger an action. Either in the same flow or other flow. See Trigger fore more information.
Waitfor is an action to stop the execution chain to wait for other actions to run or run clear before it can continue. See Waitfor fore more information.
Task
Delete is an action to clear participants from a flow. See Delete fore more information.
Dectivate is an action to disable participants in a flow. See Dectivate fore more information.
Sql is an action that is used to run one or more sql queries on a database. See Sql fore more information.
Tag is an action to create tags on participants in a flow. See Tag fore more information.
Export is a action for exporting data from a flow. See Export fore more information.
Cmd is an action to run a batch command. See Cmd fore more information.
Sweet is an action to create / update entities in Sweet CRM. See Sweet fore more information.
Market is a action for creating participants in the Sweet CRM marketing module. See Market fore more information.
Upgrade is an action that is used to pick up new versions of Sweet Automation and upgrade the system. See Upgrade fore more information.
Download is an action that can be used to download files from the web. See Download fore more information.
GDPR is a action used to clear feeds that contain sensitive information, or where information has been extended. See GDPR fore more information.
TFS Workitems is an action to create a “Work item” in Microsoft Team Foundation Server (TFS). See TFS Workitems fore more information.
E-Marketing
Email is an action to send mail via external partners or regular SMTP. See Email fore more information.
SMS is an action to send sms via external partners. See SMS fore more information.
Upload is an action to upload participants to an external partners. See Upload fore more information.
E-Marketing split is a action for distributing participants based on response from mailing. Ex. Open, Bounced, Clicked etc. See E-Marketing split fore more information.
Toolbox ETL flow¶
Loading Tasks
Excel is an action to load a flow of information that is on file. Possible formats, xlsx, csv, txt. See File fore more information.
Move
Trigger is an action that is used to trigger an action. Either in the same flow or other flow. See Trigger fore more information.
Waitfor is an action to stop the execution chain to wait for other actions to run or run clear before it can continue. See Waitfor fore more information.
Task
Delete is an action to clear participants from a flow. See Delete fore more information.
Dectivate is an action to disable participants in a flow. See Dectivate fore more information.
Sql is an action that is used to run one or more sql queries on a database. See Sql fore more information.
Export is a action for exporting data from a flow. See Export fore more information.
Cmd is an action to run a batch command. See Cmd fore more information.
Upgrade is an action that is used to pick up new versions of Sweet Automation and upgrade the system. See Upgrade fore more information.
Download is an action that can be used to download files from the web. See Download fore more information.
GDPR is a action used to clear feeds that contain sensitive information, or where information has been extended. See GDPR fore more information.
Workspace and Flow structure¶
Workspace where you build your flow.
Before we begin to build the flow, some simple explanations.
Flow boxes and actions¶
A flow is built up of actions and flow boxes. (ETL flow has only actions, no flow boxes)
Actions¶
Actions are used to load flows, move/copy or perform items on a target group in a flow.
By clicking an action, the setting dialog for the action will open up at the right.
Flow boxes¶
Flow boxes hold participants. Participants enter the top of the flow and are then divided into the flow via actions that take care of it and perform things.
Each flow box has a number to the right. That number indicates how many participants are in this box.
By clicking a box, a dialog will open up at the right. The dialog will show you the result/participants in the box. (Via the “Base template” in flow settings).
Execution and scheduling¶
All actions in a flow can be scheduled or executed in one way or another.
To the left of all actions are symbols that show how the action can be executed.
The example below shows that this action is not scheduled and can only be executed manually.
The second action has a “flash”. The flash means that the action runs automatically, i.e. as soon as the previous action above is completed.
To set whether the action should be run automatically or not, open the action and click in or out of “Auto”. Default is “Auto” checked.
The clock shows that the action is scheduled. An action can be both scheduled and set to automatic execution.
NOTE! If Auto is checked, the action will be executed if triggered by the above action regardless of whether the action itself has its own scheduling. That is, it will execute both through the previous action and its own scheduling.)
See example below:
- The “First” action is schedule for each full hour.
- The “Second” action is schedule for every monday at 1:30 pm
- The “Second” action is set to Auto!!! (flash)
Outcome for this is:
- The “First” action will run each full hour.
- The “Second” action will also run each full hour, just after the “First” action is done. (Because of “Auto”).
- And the “Second” action will also run every monday at 1:30 pm.
Conclusion: The second action can be run from two different schedules.
Schedule types¶
To set a schedule for an action, open the scheduling dialog. This can be done in two ways:
- To click the symbol to the left of the action.
- To open the action and click the “Schedule” button.
The default for all action is that they are set for manual execution.
You choose scheduling through the drop down box.
There are 6 different choices for scheduling:
Once. Only one time execution. See Once for more detailed information.
Daily. Daily execution. See Daily for more detailed information.
Weekly. Weekly execution. See Weekly for more detailed information.
Monthly. Monthly execution. See Monthly for more detailed information.
Yearly. Yearly execution. See Yearly for more detailed information.
Manual. Manual execution.
By manual execution is meant that the action can only be run manually or triggered by another system from outside.
The action is executed manually by opening the action and clicking the “Execute” button
See Manual for more detailed information.
Execution outcome¶
When an action has been run, one finds a symbol to the right of it. There are three different symbols:
Execution ongoing
Success
Failed
Below you can see that the top charge action has been successfully run.
Below you can see that the second action (split) has been executed, but failed.
By bringing the mouse over symbol, a tooltip is displayed with information about the execution and any errors.
How to build a flow¶
In this section we will go through the basics of a standard standard flow..
Once you have your newly created flow ready. Click “Edit” to open the toolbox.
To the right is now the toolbox with all the actions that can be used.
Loading action¶
Usually, you start by pulling out a loading action. Read more about the DB loading action.
In this example, we draw a “DB” loading task which is one of the most common actions.
Grab the action in the toolbox and pull it out and release it into the empty surface.
When you dragged your action, it will appear in the flow. Most of the actions also have an enclosed box behind them. The actions that have enclosed boxes get the box also a default name that can be easily changed. This box that comes with the loading action is named “Loaded”.
Note that the line between the action and the box is broken. This means that the action is not enabled.
Open the action by clicking on it. When the settings dialog is opened, the action becomes high lighted. Just to clearly show what action you opened.
This action is not configured with any selection template that will load your target audience to the flow. Choosing a selection template can be done in three ways:
- Use existing selection template by drag and drop.
- In the setting dialog for the loading action, search your selection template in the drop down box that exists.
- Create a new selection template by clicking the “Create” button.
In this particular case, we will use an existing selection template.
We will load the flow with the same selection template as we configured the flow with, under Flow Settings.
But it is very important that all selection templates used in the flow “to load”, “to distribute” or, “to export”, must contain the same field as the flows flowKey. But, there are other ways to match participants, but it requires a bit more advanced settings. Recommend to keep it simple.
Grab your selection template and drop it on the box lying under the loading action. When you drag the template over the box, te box will be high lighted and its then safe to drop it.
When you have released a selection template, the box will get the same name as the watch template. When you open the action, you will see that it is set.
Now only the action remains to be activated. Check “Active” along the right-hand side.
(Note that there are two “Active” flags. The top of the head activates or deactivates the action. The one on the right activates or deactivates the actual leg between action and box.)
Now the flow and the action is ready and can be run.
Start the flow by clicking the “Play” button.
Now the flow is loading. You can see how it is spinning to the right of it and that participants are rolling in. 100,000 at a time. (Of course, this can be set in the admin tool.)
When the action is finished, it stops spinning and you can see the green “Success” icon to the right.
Down to the right of the box is the number of loaded participants.
Now the action can be run over and over again without recharging the same participant again. Only new participants can add to the flow.
You can test by emptying the flow by clicking the “Empty” button below.
Then load the flow again through the “Play” button.
Flow box¶
By opening a box one can see the parts of the content. rather, the top 1000 of the total number of participants. The box opens up on the right side.
The result in the box is depending on the selection template that are set in “Flow settings”.
(An important thing with the contents of a flow box is that the only thing that is saved in the flow is FlowKey, in just that case it is PersonId. When a box is opened, the selection template is used to retrieve the remaining data from your data source and dress up the missing information.)
Name. Text box where you can change the name of the box.
Count. Counter shows the number of participants in the flow.
Delete. Button to delete the box. But its not allowed to delete a box that has participants in it. You must first remove participants before you delete the box.
Empty. Button to empty the box of participants. If you empty a box of participants its even been deleted from the flow.
Export. Button to export the participants to Excel. The total amount will be exported.
Advanced. Button to open up advanced settings.
Key. You can set a Key-name for the flow box. This Key can then be used via the API to get the result in it.
Description. Description that you can add to the flowbox. Maybe describes whats in it.
KPI details. Metadata that can be used for reporting etc.
Spit actions an other¶
Now that the entire target group is loaded, we want to divide it inte multiple boxes. In order to divide the target group, we use the “Split” action, the most common action of them all.
Read more about the split action: Split
Via “drag and drop” you can apply the action in the flow. Grab the action and pull out and drop it on the box you want to split. The box lights up when you can release the action.
Efter you released the action you can se it as below. The split action is now connected to the box above. After the split action there is a blue box.
In this case we want to add more boxes. From the toolbox you can grab a box at the top, and drag it out and release it on the split action. (You can only release a box on actions, not other boxes. Actions can have 0 or multiple boxes connected to them. The split action have no limitation of boxes)
Two boxes connected to the split action.
We add up to 4 boxes in this example.
Now that all boxes are extended, you can open the action. In the setting dialog to the right you can see that nothing is configured in the action.
The configuration can be done by connecting a selection template, using the TOP function or the time function or combining all three.
Now we can reuse two selection templates that we already have. These can be pulled out to resp. box. Remember that the left box has the highest priority (in the setting dialog it is the top row that has high priority). In the Setting dialog you can change the priority of the boxes by dragging and dropping them in the right priority.
Also remember that it is important that the selection templates used in a split action must contain the field that is linked to FlowKey (see Flow settings). Other considerations can be made with then more advanced knowledge is required.
Drag out the selection template and release it to a box. Rekomendation is to start with the highest priority.
After you have set the two existing templates Fund, Savings and Private and Fund, you can see that they are set in the setting dialog, but not activated.
Activate the legs and then click “EXECUTE”.
Now the action is starting to execute. It starts with the selection template that has the highest priority (far left). The selection template retrieves its participants and compares them with the box above. Those who match are moved down to the box. Those that are not matched remain in the box above. When the first leg is ready, the one that comes second in priority starts. Only activated legs are running.
When the action is complete, it can be seen that some participants remain in the box above. This means that these did not match the two active legs in the split action. The other two legs are not configured and active, they have hardly run.
Since we do not have any ready-made selection templates for the remaining legs, you can instead create them directly in the setting dialogue for the split action. Click “CREATE” to create a selection template for the leg.
When you create a template from a flow, as usual, you get the data source on which the template should be based and to give the template a name. The flow data source is preselected.
When you are inside the selection tool, you can see up to the right a button with a flow icon inside. It shows that the selection template has a connection to one or more flows. By clicking on this button, the connected feeds are displayed and you can jump in to the flow.
Unlike when you drag and drop a selection template to a box, here is no name on the box. You have to do this manually by opening the box and putting the name there.
Then open the action, activate the legs and execute the action again. The action now runs all the selection templates left to right and matches all participants in the box above. The participants who could not be matched remain.
Now we want the participants who remain in the upper box to end up in their own box. As before, pull out a new box for the action, give it a name such as “Other”. Open the split action and activate the leg. By just activating the leg without configuring anything more, that leg will have no rule and will function as a fallback where everything else ends up unless otherwise fulfilled. Execute the action again.
As you can see, the remaining participants ended up in the box without rules.
Now it turns out that one would like to have a new split action above the previous one. Ie you want to apply rules before this split.
By moving the mouse over the top of the action, a plus sign lights up. Click on this.
A drop area opens up. Here you can now drag the desired action that should end up above the action you clicked on.
Drag and drop a split action here.
Now a new action has been injected to the flow.
Add another box to the new split action. To the left leg, it should be linked to a selection template that fetches all customers who have some type of account. The second box is activated without a rule and set a different name on it.
Now it is so that all participants are already below this measure. Therefore, empty the flow and run it from the top, or “Play” button.
The flow is executed from the top. When the first action is completed, the next one starts.
Now note that the bottom right box “Other” is empty. This is because they instead ended up in the box to the right. The new rule. This box can still remain as a backup.
The box “Fund, Savings and Private” I now want to divide into 3 different boxes. I create an advanced selection template that produces a target group that has a total of 10 million on their accounts. This template is applied to the first two boxes, (left and center). But we just want to include the left of 1000 of those who have over 10 million. The box to the far right activates us without a rule, which means that these have less than 10 million.
The result of the run is shown below.
But I’m not really happy with my results. I would like to redo the rules for this distribution.
By opening up the toolbox, “Edit”, a hamburger menu lights up in the right corner of all boxes. Click on this to access actions that can be applied directly to the specific box.
In the menu that opens, select “Move” to move participants. All participants in the box will run on this action.
In the “Move” action you get the choice to move participants between boxes within the same flow or move to a completely different flow.
In this particular flow we will move up the participants in the box above. You will find the box in the drop down list by the name of the box. You can also search the box’s Id which is completely unique to the box. (Up to the left of all boxes you see the Id if you have the toolbox open.)
Klicka sedan på “EXECUTE” knappen för att köra åtgärden. Repetera denna procedur för de två andra boxarna.
Once the participants have moved up to the box above, open the action again.
Now we want to change the first rule, instead of the top 1000 we should have the top 10% random.
Then execute the action once more. Below you can see the new result.
The participants in the “Fund” box I want to move after they have been in the box for 1 hour and 30 minutes.
The formula for entering a time is ##d##h## m staring for “days, hours, minutes”.
Enter your time span and activate the action.
Then execute the action.
When the participants have been in the box above for 1h and 30 min, the action will be run, and the participants will be moved down.
The participants in the “Savings” box I want to move to the box “Other”, therefore I use a “Move” action. Pull out and release a “Move” action on the “Savings” box. Then open the action and configure it against the “Other” box, activate the action and run it.
The distribution of participants can continue indefinitely. But precisely for this flow, we feel satisfied and can connect other types of actions to really take action against the now different target groups.
Finally, we can pull out the actions that will perform different things against the target groups. E.g:
- Send e-mail
- Send SMS
- Create Activity in Sweet
- Export target group
- Create TM Activity
But keep in mind that these actions will not run as long as the flow is set for “Preview mode”.
To configure these actions and more, read more about them under Actions
After that, you can either run the flow manually or schedule it.
Good luck and have fun.