Skip to main content

Remote source user sync using SFTP and CSV

Sometimes it is required to sync workspace users based on data on some other system (CRM, Member register, other database). Howspace offers two ways to manage this kind of syncing:

CSV-format

For syncing to work, you need to have participants defined in a CSV file. Here are the requirements for CSV:

  • UTF-8 encoding

  • Comma as separator

  • Double quotes (") as an optional enclosure character

  • Double quotes should be escaped using double double quotes ("")

  • Multiple values should be separated as newlines (these can be matched against multiple choice profile fields)

  • You can name the file as you see fit

  • Always include the header row

  • Required columns are:

    • ID (unique ID from the source database)

    • Email (must be unique)

    • First name (value can be empty)

    • Last name (value can be empty)

    • Mobile (value can be empty)

  • Order of the columns doesn't matter

  • In addition to required columns, you can define custom columns that will be imported if the column name matches the profile field name on the workspace.

  • Only include active participants, we handle create, update, and delete activities as needed

  • ID needs to be unique and it is the key value we match. ID will be stored into external_id field in our database, prefixed with Sync ID and #. So if ID in CSV file is 123, then stored External ID is for example 67ab30bdda6ebafb94021e5b#123

    • This makes each imported user ID source-specific instead of relying on the raw CSV ID alone. It prevents collisions between different sync sources and to keep updates and deletions scoped correctly. It makes sure we update the right existing users, and avoid deleting users that were created by another sync source.

Example CSV

ID,Email,First name,Last name,Mobile,Role,Region,Birthdate
12345,[email protected],John,Doe,,"Manager Region Lead","EUROPE ""COMMUNITY"" REGION",1984-01-23
23456,[email protected],Peter,Pan,+358123123,Employee,"ASIA, AFRICA & AUSTRALIA",1953-02-05

Using custom profile fields for enriching user profile and for access management inside the workspace

You can use custom profile fields to bring additional data for users. Here we don't use IDs - we are matching profile field labels directly. So if you want to populate field called "Department" which is multiple choice field, then CSV header value needs to be exactly same "Department" and the value in each rows needs to match allowed values in the field. If you want to bring multiple values, use double quotes and separate them with new line.

Here is simplified example of this:

ID,Email,Department
1,[email protected],"Sales
Marketing"
2,[email protected],Sales

If you then need to restrict certain pages only for "Marketing department" in this example, you do it through user lists. You need to use dynamic userlists to connect this profile field and its value into a chosen userlist automatically.

Defining the remote source

Remote Source means connection to another service. Currently only supported connection type is SFTP. You need to define the remote source once and you can then use it for multiple workspaces.

Creating a new Remote Source in Account Manager:

  1. Click "Remote Sources" from side menu

  2. Click "Create a new Remote Source"

  3. Fill out the form. You need to know these:

    1. Host

    2. Port (defaults to 22)

    3. User

    4. Password or Private Key

  4. Click "Create a new Remote Source"

  5. You should now see Remote Source on the page

If your SFTP server has IP restrictions, these are the IP addresses that we are using to fetch data and need to be whitelisted:

  • 13.49.119.44

  • 13.49.168.26

Attach CSV from Remote Source to a workspace

After creating a Remote Source connection, you can attach a CSV file from a Remote Source to a workspace. This happens from the Account Manager => Workspaces view.

  1. Find the workspace where you want to sync users from CSV

  2. Click "..." button from the Actions column

  3. Choose "Attach Remote Source"

  4. Choose the wanted Remote Source from the dropdown and define the file path to your CSV file. The file path is relative to the user's home folder.

  5. Click "Save Remote Source"

  6. If you now navigate to Remote Sources, you should see the Workspaces on the list of workspaces with attached remote sources.

  7. You can test the sync by clicking "Sync now" from "..." menu.

  8. After the sync you can check the possible errors and warnings from "View last sync results"

Automated sync will happen now once every 24 hours. If you notice any hiccups, you can always check the results from the last sync. Also please note that sync will only update and delete users added to the workspace through the sync. It will not delete manually created admins and participants, even when they do not exist in the CSV file.

Error handling in syncs

If the are problems in CSV connection or in the data itself, you can see the errors from Customer App: Settings => Integrations => Remote Source => View Last Sync Results.

Typical errors blocking the integration completely:

Connection error: Error (ExceptionClass) opening connection: message

Opening the SFTP connection failed due to a technical error, such as a network, DNS, or socket issue.

Connection error: Unable to open connection

The SFTP connection to the server could not be established.

Connection error: Password login failed

Login with username and password failed.

Typical errors that cause skipping the user (we continue processing the other users):

Records must have: ID, Email

The row is skipped because it is missing one of the required fields: ID or Email.

Invalid e-mail address

The user is skipped because the email address is not in a valid format.

Record has incorrect number of fields

The row is skipped because it does not contain the same number of columns as the header row.

Typical warnings (we still update the user, but with some limitations):

Unknown profile field value: value

The user can still be imported, but this profile field value is not recognized and is ignored.

Multiple values for field: field name

The field contains more than one value even though only a single value is allowed, so the field is not updated as expected.

Field X (Field name) is unknown

The CSV contains a column that is not recognized as a supported field, so that column is ignored during import.

Did this answer your question?