Create a Source System Connection using the CI Sync Agent Config Utility with a SQL RecVer DB
Infrastructure SME
CI Sync Admin
3
Mongo
Or, Create a Source System Connection using the CI Sync Agent Config Utility with a MongoDB RecVer DB (if supported)
Infrastructure SME
CI Sync Admin
4
Finalise Settings in the CI Sync SaaS UI
CI Sync Admin
5
Perform Updates in ServiceNow (if required)
ServiceNow Admin
Task 1: Prepare VMware vSphere for use with CI Sync
The follow are required to prepare VWware for use with the CI Sync Agent.
Prepare your vSphere Environment to allow CI Sync Agent to read data
Create a login in your vSphere Client environment. Syncfish recommend a dedicated account is used (i.e. one created specifically for the CI Sync Agent).
You will need the username and password of this login to be used by CI Sync Agent to authenticate access to your VMware environment.
The dedicated login you just created will need the Read-only role. Syncfish recommend the account has least privilege access and meets your internal security standards for an integration account to access your vCenter.
You will need the URL of your VMware vCenter to complete a connection setup to your vCenter environment required further down on Task 3.
Note in this example screenshot, the URL on top of the vSphere Client screen and the dedicated user with Read-only role.
Please reach out to your Syncfish contact person for additional assistance with the source system setup/configuration.
Task 2: Understand the CI Sync RecVer Database
This “task” is solely about understanding the purpose of the CI Sync RecVer database and making decisions on where to host it and how the CI Sync Agent will authenticate to it.
You may also need to make a decision on which database technology to use for the RecVer Database. The CI Sync Agent supports both MS SQL and MongoDB for the RecVer database (however there are some restirctions on which database technology you can use depending on the source system itself).
Understanding the above topics and making the relevant decisions before you create the source system connection (via the CI Sync Agent Config Utility) will make it quicker/easier to execute the remaining tasks in this guide.
Click to read a comprehensive explanation of the CI Sync RecVer database (and many close related topics)
Firstly, Context about the purpose of CI Sync RecVer Database
The RecVer database is a small checksum database used by CI Sync to support the delta synchronization functionality (i.e. the functionality of CI Sync to only sync those records which have changed since the previous sync job).
Each source connection you create (using the CI Sync Agent Configuration Utility) requires its own RecVer database. That is, each RecVer database is specific to each individual source system connection.
Second, options for the RecVer database technology and the hosting server/VM
IF the source system technologyis based on Microsoft SQL (such as SCCM, Lansweeper, etc) THEN the CI Sync RecVer database must be Microsoft SQL Server. In this circumstance:
The RecVer database must be hosted on the same SQL Server as the source system database itself.
IF the source system technology is NOT based on Microsoft SQL (such as Intune, Azure, VMware, etc) THENthe CI Sync RecVer database can be hosted on either Microsoft SQL or MongoDB. In this circumstance:
For a SQL based RecVer SQL database, the RecVer DB can be hosted on any SQL server you chose as long as it is “nearby” the VM running the CI Sync Agent (i.e. the Windows Service) itself. Note: “Nearby” means having low latency between the CI Sync Agent VM and the SQL Server hosting the RecVer DB).
For a MongoDB based RecVer database, the MongoDB database and CI Sync Agent must both be hosted on the same VM.
Third, SQL authenitcation options
If you are using a MS SQL source system (such as SCCM or Lansweeper) and therefore using MS SQL to host the RecVer database you need to decide how the CI Sync Agent will authenticate to SQL.
The CI Sync Agent (i.e. the Windows Service) requires read/write access to the RecVer database. The CI Sync Agent also requires read access to the relevant SQL based source system (e.g. SCCM, Lansweeper, etc).
When you use the CI Sync Agent Config Utility to create the source system connection (so you can ultimately run sync jobs from that source system) you have the following options for authentication between the CI Sync Agent (i.e. the Windows Service) and the RecVer DB and source system iteself:
Integrated Security: This option uses the Windows User Account you created for the CI Sync Agent Windows Service.
SQL Native Login: This option uses a native SQL username and password you enter into the CI SYnc Config Utility when you are creating the source system connection.
Note: In either case (Integrated Security or SQL Native Login), the account/login only needs least privileged access to the SQL DBs. Permissions are set either by your DBA (through a manual process) or the Config Utility (through the automated process) as explained further below.
Finally, options for the SQL RecVer DB creation and setting of SQL permissions
If you are using a MS SQL source system (such as SCCM or Lansweeper) and therefore using MS SQL to host the RecVer database you have two options for for creating the SQL based RecVer DB and for setting pemissions to it (the RecVer DB) and to the source system itself.
Your options are as follows:
MS SQL Setup Option 1: In this option you use the CI Sync Config Utility to automatically create the RecVer database and grant the CI Sync Agent (i.e. the Windows Service) access to the source system SQL database.
MS SQL Setup Option 2: In this option you engage a SQL DBA to manually create the RecVer database and grant the CI Sync Agent (i.e. the Windows Service) access to both the RecVer DB and also to the source system SQL database. The DBA will perform these actions if you don’t have permissions to the SQL server (and therefore you aren’t able to use the CI Sync Config Utility as described in Option 1 above).
Task 3 (SQL): Create a Source System Connection using the CI Sync Agent Config Utility with a SQL RecVer DB
As explained in the preceding “task” there are two options to facilitate the creation of the RecVer database and assigning permissions to it for the CI Sync (Agent). The table below elaborates the two options.
Option
Description
Details
MS SQL Setup Option 1
Automatically using the CI Sync Agent Config Utility
This option requires you to have the sysadmin (sa) role in the SQL Server that will host the RecVer database.
If you have sa credentials to the SQL Server that will host the RecVer database you can use the CI Sync Agent Config Utility to create the CI Sync RecVer database.
For this option use the Option 1 instructions below.
MS SQL Setup Option 2
Manually via a SQL Database Administrator (DBA)
If the person executing the CI Sync Agent Config utility does not have the sysadmin (sa) role in the SQL Server, you can have your SQL DBA perform a manual setup of the RecVer database.
For this option you should contact your DBA nowand ask them toperform the manual setup tasks described in the Option 2 instructions below.
Once the DBA has completed their allocated tasks you will continue with the Option 2 instructions and use the CI Sync Agent Config Utility to connect to the RecVer database the DBA has created for you.
Expand the instructions below for either Option 1 or Option 2.
Option 1: Use the Config Utility to automatically create the RecVer database
Expand the instructions below to let the CI Sync Config Utility automatically create the CI Sync RecVer database (and set the required permissions to both the RecVer and Lansweeper SQL DBs).
Click to expand the instructions to setup the source connection with a SQL RecVer using Option 1
Run the “CISynchronizerAgent Config Utility” from the Windows Start Menu.
Within the CI Sync Agent Configuration Utility, navigate to the Source Systems tab, select VMware and click the Add button.
The “Add Connection” form is shown (specific to the source system).
In the section titled “Settings for the CI Sync Windows Service” update the values as follows:
Name: Enter a friendly name that describes this source system connection.
If you will have multiple vCentres being read by CI Sync make sure you use a meaningful name (i.e. a name that identifies the specific vCentre this CI Sync connection relates to).
URL: Enter the URL to the VMware vCentre as noted in the task 1.
User: Enter the username of the dedicated user created in the task 1.
Password: Enter the password for the user created in the task 1
Timeout: Syncfish recommend initially setting this to 60 seconds.
Move to section titled “RecVer DB Settings”
Update the “RecVer DB Settings” values as follows:
RecVer Type: Set this to SQL Server.
In the sub-section titled “SQL Server RecVer Settings” update the values as follows:
SQL Server: Enter the SQL Server path in the format: server\instance.
Otherwise, uncheck this checkbox and enter the User ID and Password for a Native SQL Login provided to you by the DBA. (Note: Syncfish do not recommend the SQL Login have sysadmin rights). Note: These authentication settings control how the CI Sync Agent Windows Service authenticates with your SQL Server during ongoing operation of the agent (I.e. when sync jobs are running).
Bypass setup, I have manually setup the databases: Do NOT check this checkbox (because the Config Utility will be creating the RecVer database for you).
Existing RecVer Database: Do NOT check this checkbox (as the Config Utility will create the RecVer database for you).
Note: These RecVer tick boxes (Bypass setup and Existing RecVer) control whether the CI Sync Config UI initiates the setup of the RecVer database (or not if your DBA is doing the SQL setup manually).
RecVer Database: Greyed out so can be ignored.
DB Timeout (Secs): Sets the connection and statement execution timeouts. Syncfish recommend 60.
Locate the “Setup & Test” button
Do the following:
Click the “Setup & Test” button.
When the button “Setup & Test” is pressed, the CI Configuration utility will prompt if the user running the utility wants to use Integrated Security or SQL Server security to perform the automated SQL setup and configuration steps.
Informational Note
The above values do not control the ongoing running of the CI Sync Agent, they are solely used by the CI Sync Config UI to perform the RecVer setup.
The user will be required to have admin role in SQL Server to be able to setup the RecVer
Guidance Notes
In the background the Config Utility performs the following SQL setup and configuration steps:
The utility creates a SQL Login for the Windows User Account you setup for the CI Sync Agent Windows Service if you selected “Integrated Security” in the section “Settings for the CI Sync Windows Service”, the Config Utility. Or, if instead you are using Native SQL Login (instead of “Integrated Security”) then Config Utility uses the Native SQL Login details you have supplied and performs the following step to grant the Login access to the relevant databases with the permissions noted below.
The utility creates the RecVer database, creates a SQL user in the RecVer database for the Login used by the CI Sync Agent and grants the SQL user the agent_role role.
Note: Depending on whether you selected “Integrated Security” or not, the SQL user is either (a) the Windows User/Service Account (created as a SQL Login in point 1 above) or (b) the native SQL login account.
Note: If you are using a Native SQL Login (not Integrated Security) with higher privileges than noted above (e.g. if the account had SQL sysadmin) then the Config Utility will not be able to set the SQL roles noted above. Instead, the Config Utility will not touch the permissions and instead will leave the existing/higher privileges as is. This is not required and not recommended by Syncfish.
If all is setup correctly you will see “Ok” against each of the setup tasks. If you see errors recheck the values you entered on the form.
After a successful test has been completed, click the Save button.
Finally, click Yes to register this new connection (via your CI Sync Agent) with your customer specific CI Sync SaaS instance.
Assuming no errors you will see a confirmation message. Click OK and the new connection will appear in the Connections List of the CI Sync Agent.
You have now completed the setup of the source connection in the CI Sync Configuration Utility and registered it as a source connection in the CI Sync SaaS application.
Option 2: Use your SQL Database Administrator (DBA) to manually create the RecVer database in advance
Expand the instructions below if your SQL DBA will manually setup the RecVer database (and set the required permissions to both the RecVer and Lansweeper SQL DBs).
Click to expand the instructions to setup the source connection with a SQL RecVer using Option 2
Ask you SQL DBA to perform the tasks listed in the table below.
Task
DBA Task (in SQL Server)
Additional Notes
Task 1
Either register a Login (which represents the Windows User Account used by the CI Sync Agent Windows Service)
Or create a SQL Login (for the CI Sync Agent to use).
If you are using Windows “Integrated Security” between the CI Sync Agent and the SQL Server, then the CI Sync Agent Windows Service account needs to be registered within SQL Server as a Login.
Alternatively, if you are using Native SQL authentication between the CI Sync Agent and the SQL Server, then a Login will need to be created. In this case your DBA will need to provide you with the SQL User ID and password details so you can enter them into the Config Utility UI as explained further below.
Task 2
Edit then Execute the SQL Script provided by Syncfish located here. The script creates the Syncfish RecVer database.
The script is: Cisee-recver-create-script.sql
Notes for the DBA when running the script:
The script has a placeholder called $(database_name) as the replacement for the RecVer database name.
For this source systemSyncfish recommend “cisee_recver_vmware” as the RecVer database name.
To run the setup script using the sqlcmd utility, the database_name parameter needs to be passed in using the -v switch:
If running the script in SSMS, the value $(database_name) (including the $ and brackets) needs to be replaced with the name for the recver database.
Task 2
Edit then Execute the SQL Create Script provided by Syncfish located here. The script creates the Syncfish RecVer database.
The script is: cisee-recver-create-script.sql
Notes for the DBA when running the script:
The script has a placeholder called $(recver_database_name) as the replacement for the RecVer database name.
For VMware vSphere as the source system Syncfish recommend “cisee_recver_vmware” as the RecVer database name.
To run the setup script using the sqlcmd utility, the recver_database_name parameter needs to be passed in using the -v switch:
> sqlcmd -I -S localhost -v recver_database_name=" cisee_recver_vmware" -i "cisee-recver-create-script.sql"
Or, if running the script in SSMS, the value $(recver_database_name) (including the $ and brackets) needs to be replaced in the .sql file with the name for the recver database.
The DBA may notice the script includes has a placeholder called $(source_database_name) which is only required when Lansweeper is the source system. This placeholder can be ignored for all other (non-Lansweeper) source systems.
Task 3
Edit then Execute the SQL Upgrade Script provided by Syncfish located here. The script creates the Syncfish RecVer database.
The script is: cisee-recver-upgrade-script-3.1.1.sql
Notes for the DBA when running the script:
The script has a placeholder called $(recver_database_name) as the replacement for the RecVer database name.
For VMware vSphere Syncfish recommend “cisee_recver_vmware” as the RecVer database name.
To run the setup script using the sqlcmd utility, the database_name parameter needs to be passed in using the -v switch:
Or, if running the script in SSMS, the value $(recver_database_name) (including the $ and brackets) needs to be replaced with the name for the recver database.
Task 4
Map the Login as a user in the RecVer database and grant the user the role_agent role.
Map the user and grant the role to the RecVer database created by the SQL script above.
Once your DBA has completed the steps in the table above, use the CI Sync Agent Configuration Utility to setup the source connection and pointing to the RecVer database created by the DBA. See below for instructions on using the CI Sync Agent Configuration Utility.
Run the “CISynchronizerAgent Config Utility” from the Windows Start Menu.
Within the CI Sync Agent Configuration Utility, navigate to the Source Systems tab, select VMware and click the Add button.
The “Add Connection” form is shown (specific to the source system).
In the section titled “Settings for the CI Sync Windows Service” update the values as follows:
Name: Enter a friendly name that describes this source system connection.
If you will have multiple vCentres being read by CI Sync make sure you use a meaningful name (i.e. a name that identifies the specific vCentre this CI Sync connection relates to).
URL: Enter the URL to the VMware vCentre as noted in the task 1.
User: Enter the username of the dedicated user created in the task 1.
Password: Enter the password for the user created in the task 1
Timeout: Syncfish recommend initially setting this to 60 seconds.
Move to section titled “RecVer DB Settings”
Update the “RecVer DB Settings” values as follows:
RecVer Type: Set this to SQL Server.
In the sub-section titled “SQL Server RecVer Settings” update the values as follows:
SQL Server: Enter the SQL Server path in the format: server\instance.
Otherwise, uncheck this checkbox and enter the User ID and Password for a Native SQL Login provided to you by the DBA. (Note: Syncfish do not recommend the SQL Login have sysadmin rights). Note: These authentication settings control how the CI Sync Agent Windows Service authenticates with your SQL Server during ongoing operation of the agent (I.e. when sync jobs are running).
Bypass setup, I have manually setup the databases: Check this checkbox (because your SQL DBA has already manually created the RecVer database and set the relevant permissions for you).
Existing RecVer Database: Check this checkbox (because your SQL DBA has already manually created the RecVer database for you
Note: These RecVer tick boxes (Bypass setup and Existing RecVer) control whether the CI Sync Config UI initiates the setup of the RecVer database (or not if your DBA is doing the SQL setup manually).
RecVer Database: Enter the name of the RecVer database your DBA created. If your DBA used the suggested name (described on the previous page) it will be called cisee_recver_vmware.
DB Timeout (Secs): Sets the connection and statement execution timeouts. Syncfish recommend 60.
Locate the “Setup & Test” button
Do the following:
Click the “Setup & Test” button.
If all is setup correctly you will see “Ok” against each of the setup tasks. If you see errors recheck the values you entered on the form.
After a successful test has been completed, click the Save button.
Finally, click Yes to register this new connection (via your CI Sync Agent) with your customer specific CI Sync SaaS instance.
Assuming no errors you will see a confirmation message. Click OK and the new connection will appear in the Connections List of the CI Sync Agent.
You have now completed the setup of the source connection in the CI Sync Configuration Utility and registered it as a source connection in the CI Sync SaaS application.
Task 3 (Mongo): Or, Create a Source System Connection using the CI Sync Agent Config Utility with a MongoDB RecVer DB (if supported)
Click to expand the instructions to setup the source connection with a MongoDB RecVer
Run the “CISynchronizerAgent Config Utility” from the Windows Start Menu.
Within the CI Sync Agent Configuration Utility, navigate to the Source Systems tab, select VMware and click the Add button.
The “Add Connection” form is shown (specific to the source system).
In the section titled “Settings for the CI Sync Windows Service” update the values as follows:
Name: Enter a friendly name that describes this source system connection.
If you will have multiple vCentres being read by CI Sync make sure you use a meaningful name (i.e. a name that identifies the specific vCentre this CI Sync connection relates to).
URL: Enter the URL to the VMware vCentre as noted in the task 1.
User: Enter the username of the dedicated user created in the task 1.
Password: Enter the password for the user created in the task 1
Timeout: Syncfish recommend initially setting this to 60 seconds.
Move to section titled “RecVer DB Settings”
Update the “RecVer DB Settings” values as follows:
RecVer Type: Set this to Mongo Db.
In the sub-section titled “SQL Server RecVer Settings” update the values as follows:
MongoDb Connection: Enter the Mongo Db instance path in the format: mongodb://server.
Bypass setup: Do NOT check this checkbox unless you have been asked to do so by Syncfish (in most cases the Config Utility will be creating the RecVer database for you).
Existing RecVer Database: Do NOT check this checkbox unless you have been asked to do so by Syncfish (in most cases the Config Utility will be creating the RecVer database for you).
RecVer Database: This field will be greyed out in most cases (unless one of the above two options have been ticked). In most cases there is no existing RecVer database and therefore the Config Utility will automatically generate the name of the database when you click the “Setup & Text” button as part the next step in these instructions.
Locate the “Setup & Test” button
Do the following:
Click the “Setup & Test” button.
If all is setup correctly you will see “Ok” against each of the setup tasks. If you see errors recheck the values you entered on the form.
After a successful test has been completed, click the Save button.
Finally, click Yes to register this new connection (via your CI Sync Agent) with your customer specific CI Sync SaaS instance.
Assuming no errors you will see a confirmation message. Click OK and the new connection will appear in the Connections List of the CI Sync Agent.
You have now completed the setup of the the source connection in the CI Sync Configuration Utility and registered it as a source connection in the CI Sync SaaS application.
In the CI Sync UI, navigate to Settings > Connections.
Find the new source system connection you just added in the list of Source Connections (the screen shot above is a sample only).
Find your specific Source System Connection in the list and click the Update hyperlink (on the right hand side of the screen).
The connection Settings Form is presented. Update as follows:
Enter an Alias (optional) - the alias is only used in the CI Sync SaaS UI to show a friendly name in various UI forms.
Set the Environment/s the new source connection can be used for.
In most cases a Source System Connection is used for both Test and Production sync jobs (as distinct from the Destination Connections which can only be either Test or Production).
The Environment value is used to filter the connections dropdown list when you are creating a sync job.
While you are on this page you can/should check whether there are any connection specific settings you may want to adjust either now or at some point in the future. Connection specific settings (or just Connection Settings) allow you to override the default data sync rules for your CI Sync instance.
Read the following details to understand more about CI Sync Connection Settings:
Scroll further down to the Additional Settings section on the page to see any available Connection Settings. Below is an example of the sorts of settings you might notice.
The settings are specific to each source connection so the screen shot is an example only.
Syncfish recommend you read the following documentation before overriding any of the default settings:
Read the CI SyncDefault Configuration Guides. The pages in that tree provide comprehensive information about the default behaviour of the CI Sync data sync rules, the options available for overriding those rules and typical reasons why you might want to do this.
Read Understanding the use of CI Sync Connection Settings. This page explains how the Connection Settings should be used, how to modify settings via the CI Sync UI and how to test any setting changes in non-production prior to production.
Finally, if you are ready to modify any of the Connection Settings, visit Connection Setting Guides and locate the specific Source System page/s in that tree. The individual pages in that tree provide detailed information about each setting.
After making any changes on this page, scroll to the bottom of the page, Check the consent checkbox and Click the Save connection button.
You have now completed all tasks to add your new Source Connection in the CI Sync Agent.
Task 5: Perform Updates in ServiceNow (if required)
In this section your ServiceNow SME will assess various updates to ServiceNow to support this CI Sync connector:
Task 5a: Assess if the CMDB CI Class Models plug-in is required
Task 5b: Assess if additional permissions are required
Task 5c: (Optional though recommended) Assess your ServiceNow CI forms and update to include additional Related Lists
Guidance Note
Syncfish recommend the person setting up the source system described in this guide discusses this particular task with their ServiceNow system administrator.
A ServiceNow administrator will need to perform these steps.
Syncfish recommend following these instructions in your non-production ServiceNow environment for testing synchronization jobs.
Only once exhaustive testing in non-production is complete, repeat this process in your ServiceNow production environment.
Task 5a: Assess if the CMDB CI Class Models plug-in is required
The recordsets available to the CI Sync VMware vSphere connector do not require the ServiceNow CMDB CI Class Models plug-in. Therefore you can ignore/skip this task.
Task 5b: Assess if additional permissions are required
Context
CI Sync writes VMware Custom Fields to the cmdb_key_value table in ServiceNow.
The standard/out-of-the-box roles provided by ServiceNow (and recommended by Syncfish during S6 - Configure your ServiceNow for CI Sync) do not provide access to the cmdb_key_value table. Therefore, the CI Sync Integration User account created during S6 - Configure your ServiceNow for CI Sync requires additional permissions to write to the cmdb_key_value table.
Syncfish provides a ServiceNow updateset to prepare your ServiceNow instance for CI Sync. The updateset does the following:
Creates a read/write ACL on the cmdb_key_value table.
Applies the ACL on the cmdb_key_value table and assigns the ACL to the ServiceNow role called “Asset” (which is one of the roles granted to the CI Sync Integration Account created during S6 - Configure your ServiceNow for CI Sync).
Instructions
Follow these steps to apply the updateset provided by Syncfish:
Login to your ServiceNow instance with Admin permissions.
Open a browser and navigate to your ServiceNow instance
In the left nav menu search for “Retrieved Update Sets” and click to open
Right click on the column heading row and select “Import XML”
Select “Choose File”
Select the downloaded file “cisync-cmdb-key-value.xml”
Click to open the Update Set
Click “Preview Update Set”
If there are no preview errors, Click “Close”.
Click “Commit Update Set”.
Your ServiceNow instance is now ready to receive VMware Custom Fields from VMware via sync jobs from CI Sync.
Task 5c: (Optional though recommended) Assess your ServiceNow CI forms and update to include additional Related Lists
Context
CI Sync populates various child tables (related lists) associated with parent CIs. The following table shows the Related Lists (per CI Class) populated by the CI Sync VMware vSphere Connector.
CI Class
Related List (i.e. friendly name)
Related List Name as it appears in the ServiceNow UI when adding it to a CI Form
Below are the steps to modify a ServiceNow CI form to expose a new Related List.
Login to your ServiceNow instance with Admin permissions.
Navigate to any CI in the relevant CI Class (i.e. one/all of those listed in the table in the Context section above). For example, navigate to a Windows Server CI).
Right-click in the heading area of the form, then click Configure and then Related Lists from the sub-menus.
Identify the Related List you want to expose on the CI form using the table in the Context section above.
Find the Related List in the left hand column which lists all Available Related Lists.
Click the Related List and then click add (the selection arrow) to move the item to the Selected column and then click Save.
Repeat for each additional CI Class listed in the table in the Context section above.