File Watcher Monitoring

IT-Conductor File Watcher facilitates application integrations by tracking file exchanges between business partners, clients, providers, etc. This monitoring mechanism enables seamless communication through file sharing, where entities post files into designated directories for application consumption.

The file watcher functionality covers SMB (Samba) File Server environments, enabling users to monitor shares, directories, files, their existence, size, and modification time. By setting up alerts, users can ensure that specific conditions regarding the files, such as timely arrival, are met. Users can trigger automated processing and enable event-driven integrations by setting up recovery actions. This proactive monitoring and automation approach enhances operational efficiency and helps maintain compliance and data integrity across integrated systems.

Pre-Requisites

Set up a file server for file management and watcher in IT-Conductor. See File Server Monitoring for more details.

Configure File Watcher Monitoring in IT-Conductor

To configure the file watcher monitoring in IT-Conductor, follow the instructions below.

Add New File Retriever

Once the file server has been added, perform the following steps to set up folders and files periodically to pull from the file server into IT-Conductor.

1. Visit service.itconductor.com and enter your login credentials.

2. Navigate to the file server node and click Retrievers.

Figure 1: File Server Node in Service Grid

1. Click to add a new retriever.

Figure 2: Existing Retrievers in the File Server

1. Fill out all the necessary information in the Create File Retriever wizard. Once completed, click to add the system.

Figure 3: Create File Retriever Wizard

• Name - refers to the name given to the retriever.

• Description - refers to any relevant information about the retriever being added.

• Repeat interval - refers to the frequency with which each retriever pulls files into IT-Conductor. By default, this is set to every 15 minutes. However, depending on the sensitivity of the information, this can be adjusted as needed.

• Share - refers to the share that will be pulled into IT-Conductor.

• Folder - refers to the folder's name inside the share that will be pulled into IT-Conductor.

• File Name/Mask - refers to the file names that will be pulled into IT-Conductor. You can use regex in this field. (Optional)

After saving, it’ll take a few minutes for IT-Conductor to pull the files in that share and folder.

Figure 4: File view in IT-Conductor

Add New File Watcher

By setting up the file watcher, you specify how folders and files periodically pulled from the file server are monitored.

1. On the file server node, click File Watchers.

Figure 5: File Watchers

1. Click Create New Object to create a new file watcher.

Figure 6: Create a new File Watcher

1. On the following screen, fill out the following details

• Name - refers to the name assigned to this file watcher.

• Description - refers to any relevant information about the file watcher being added.

• Application - refers to the file server.

• File Mask -This is an optional file selector - a regular expression (Regex) instruction where you specify what files will match the criteria. In the simplest form, specify the full path of the file, but make sure you escape special characters such as []().:\^$ with '\' as: \[, \], \(, \) etc. For more advanced options, please refer to online regex tutorials and testers such as Free Online Regular Expression Tester - FreeFormatter.com

1. Click to save.

Figure 7: Create File Watcher Wizard

1. Click the recently created file watcher. Then click Threshold Overrides to create an override to specify what the file watcher will monitor.

Figure 8: File Watcher Activity Chart

Note: The overrides in the selected file watcher will apply only to the files pre-selected by the monitor based on the corresponding file mask.

1. On this page, you may see the previously created overrides for monitoring. Click to create a new override, or click to create an override from an existing template. In this example, we’re creating a new override.

Figure 9a: Creating a New Override

1. Fill out all the necessary information in the Create New Override wizard. Once completed, click to add the new override.

Figure 9b: Creating a New Override

• Name - refers to the assigned name for the override being added.

• Description - any relevant information about the override being added.

• Object Criteria - refers to the specific attributes that will be monitored. If you add the folder path in step 3, it will appear here. Under Object Criteria, you may specify what you want to monitor in this path. To add more criteria, click on the Add New Row , then fill in the following fields:

  • Name - refers to the exact criteria that you can monitor. You can choose one of the available criteria from the drop-down menu.

  • Oper - refers to the operator validation (=, !=, >, <, in, regex, NULL, etc).

  • Value - refers to the exact value that will be monitored. This is an open field where you can specify file names or formats for monitoring.

In the example above, we add the object criteria Path, and the operator '=', which means that the override will search for files within the folder path that we add.

Figure 9c: Creating a New Override

• Scheduling - refers to the section where you can specify when the override will perform validation. You may choose to run the override on a specific day and time. If you don't specify a day, it will run daily at the indicated time. Alternatively, you can assign a pre-existing schedule from the dropdown menu.

  • Start - refers to the time when the override will start monitoring.

  • End - refers to the time when the override will stop monitoring.

• Aggregation - refers to the metrics used to define the aggregation values.

  • Aggregation interval - defines the period within which the files are collected and added to the file server.

  • Consecutive interval - refers to the regularity or frequency of occurrences within a defined amount of minutes.

  • Aggregation - refers to the function that will be applied, such as sum, average, count, minimum or maximum.

The Scheduling and Aggregation sections tell the override when to look for files. The example above specifies a time window between 6:30 am and 7:00 am. During that time, the override will search for files added to the server in the previous 360 min, as specified in the Consecutive Interval field below.

Figure 9d: Creating a New Override

• Thresholds - refers to the metrics used to define the threshold values.

  • Warning Value - refers to the value determining if the threshold was reached. If this field is blank or zero, it means that there’s no data.

  • Warning Operator - refers to the operator validation (=, !=, >, <, in, regex, NULL, etc).

  • Warning Severity - refers to the severity status that will be triggered if the validation reaches the Warning Value.

• Alerting - refers to the section where you can indicate when the users will receive the alerts.

  • Alert On - refers to the status that will trigger the alert and notify the users. This is usually set to “Warning”.

  • Alert Message - refers to the message that the users will see. This message is customizable by the user.

Threshold and Alerting sections tell the override when to send a message to the user. This example specifies that if there have been 0 posted files in the server, the status of the override will turn to Warning, and if the status is Warning, an alert will be sent to the user.

Note: The more criteria you add, the more specific the override is and the higher the precedence.

1. Click to save.

Figure 10: Existing File Watchers

Note: The status will remain on Ready for a few minutes. Refresh the panel if needed. If there are configuration issues, such as incorrect connection parameters or an invalid account, the status will remain Ready. Click to display the logs and troubleshoot. If everything is configured correctly, the status will change to In Progress.

Monitor Files in IT-Conductor

Locate the previously configured file server under the File Servers node in the service grid to monitor the files pulled from the file server. Click Files to view all the files added to that share.

Figure 11: Pulling Files from the File Server

By clicking on any of the files, you can see the following information from each file:

• Description - refers to any relevant information about the file.

• File name - refers to the name of the file and its extension.

• Path - refers to the path’s full size, including the share and folders before it.

• Size - refers to the file size.

• Last modified - refers to the time and date when the file was last modified or added to the server.

Figure 12: File information

File Watchers

To monitor the existing file watchers, locate the File Watchers node in the service grid.

The number displayed next to the file watcher indicates the number of alerts generated by the override. Click to inspect the activities.

Figure 13: File Watcher View in Service Grid

By enlarging the chart, you can see the activity of the override. By clicking on the green bars, you can see which files have arrived recently and at what time. The image below is an example of 3 files arriving in the folder path at 8:30pm.

Figure 14: Reading the override charts

Alternatively, the following image represents an alert sent to the user’s email because the override didn’t detect any files during the consecutive interval period.

Figure 15: Reading File Watcher Alerts