Knowledgebase

Yes, the SpaceObServer data can be integrated by accessing the database using the database connectors offered in PowerBI. Examples for this can be found in the installation directory of SpaceObServer under 'C:\Program Files\JAM Software\SpaceObServer\Demos\PowerBI' along side a documentation on how to use it.

Due to how the data is structured in the database and how PowerBI uses data there are some limitations on what can be achieved. These are also explained in the documentation.

This error is caused by an operation that is part of the deduplication process and tries to consolidate the different permissions of the duplicates.

Hardlinks all share the same permission settings, so the deduplication process tries to ensure that access rights that a user previously had will still apply after a deduplication.

Here's an example:
Assuming you deduplicate two duplicates "file A" and "file B".
File A has "read only" access for user X
File B has "read and write" access for user X
File B will be replaced by a hardlink that points to File A
The "larger" set of access rights from File B has to be applied to File A as well

In summary: When one of the files is replaced with a hard link to the other, the process will try to merge the permissions of the underlying data. In the example above, the consolidated access rights would be "read and write" for user x. 

However, this operation is only successful, if the current user is allowed to change access rights for that particular file, or for files in the current folder. It is important to keep in mind that having the access right to change or write a file does not automatically include the access right to change the permission settings of that file.

If you receive an error message "Failed to merge permissions: Access is denied", then this means your current user is not allowed to change these permission settings and no deduplication will be performed for this set of duplicates. 

This can be the case either if he does not have an "Allow" entry for the operation "Change permissions", or if an explicit "Deny" entry exists for that. It is also important to note that a "Deny" entry will always overrule an "Allowed" entry, if both are set.

There can be several possible causes:

• The last indexing run took place before the file was created or modified, so the current content of that file isn't yet in the index.
• The indexing service may not have had sufficient permissions to access the file.
• The file's type isn't (yet) supported - see https://knowledgebase.jam-software.com/7719
• The file is invalid, corrupted or otherwise unreadable.
• DataCentral ran into a problem when extracting the file (for example, because it's too large or has an unexpected structure).
• DataCentral does not index files that have any of these attributes: Hidden, Temporary, NotContentIndexed
• If the file was indexed successfully, but UltraSearch still doesn't find your search term, it might be due to the term itself - or it could be that less important content (for example, HTML tag names) isn't included in the index.

No, DataCentral does not transfer any sensitive data out of your company network. All data is stored in an on-premises database on your premises.

No cloud services are required or used when using DataCentral.

DataCentral has access control implemented.

Although, there might be files indexed that are not accessible by some users, DataCentral automatically only delivers search results the specific UltraSearch user has access to.

No. There are no built-in limitations regarding amount of files/data. However infrastructure and SQL server may impose limits. The number of scanned elements therefore has an impact on the speed of creating reports. Some reports might scale worse than others depending on the number of files scanned. We expect up to 500 mio files per scan to work given that the SQL server has sufficient resources and the connection to the storage system is good with low latency.

When dealing with very large folders/drives, the infrastructure SpaceObServer is hosted on could cause performance bottlenecks. Please find our recommendations here.

Usually, you need 1 license to run SpaceObServer and DataCentral on a server + n licenses of UltraSearch Professional for n people that can search the DataCentral indexes.

The amount of data in the indexes and the number of searches is unlimited.

The size of the index varies and depends on many factors:

• the number of actually indexable files (xlsx and txt are indexable... exe, images & videos are not)
• the size of the files

For an average drive, the required space ranges between less than 0.01% of the target to 1%.

For a very high ratio of textual content, up to 30% of the target size might be necessary, but this is quite rare.

Please open UltraSearch, navigate to the DataCentral connection configuration settings and press the button "Test connection". If the connection check fails, please continue with these options:

1. Verify service availability and network connectivity
   Make sure the "SpaceObServer DataCentral" service is running on the SpaceObServer host system.
   Open https://hostname:port/healthz or http://hostname:port/healthz. The endpoint must return "Healthy".
   Make sure the DataCentral connection is not blocked by a firewall. The default port is 5149.
   In SpaceObServer, open Options > Index Service and verify that the DataCentral server is enabled and uses the same port that is configured in UltraSearch.

2. Verify the certificate used by DataCentral
   In SpaceObServer, under Options > Index Service, check which certificate is currently configured for the DataCentral service.
   Compare the certificate thumbprint shown there with the certificate or fingerprint configured in UltraSearch.
   In UltraSearch, certificate settings can only be changed in administration mode.
   If UltraSearch was installed without explicit certificate parameters, it validates the DataCentral certificate by fingerprint by default and stores the fingerprint from the first connection for later communication.

3. Verify authentication and SPN configuration
   Try out connection to the IP adress of the SpaceObServer DataCentral host in UltraSearch. If the connection works with the IP address but not with the server name, the problem is typically related to Kerberos/SPNs. UltraSearch uses NTLM when connecting by IP address and Kerberos when connecting by host name.
   If the connection check with the hostname returns something like "the user is not authenticated" / "cannot access the resource", you can
   1) use NTLM (use the IP address) as authentication method (not recommended because of security reasons)
   2) making Kerberos work by adding the required SPNs to the user account running the DataCentral service (via your ActiveDirectory).
   If DataCentral runs under a dedicated domain account, register the HTTP SPNs on that service account (not on the computer account).
   Use setspn -L <account> to review the current SPNs. Typical examples are HTTP/servername and HTTP/servername.domain.tld.
   Modifying SPNs requires the appropriate permissions on the account object.
   Use setspn -S <SPN> <account> to add a missing SPN.
   
   

If the connection check works, please follow these steps:

1. Scan/Index existence
   If the search returns no results, verify that the folder you are searching in has already been scanned and indexed by SpaceObServer/DataCentral.
   For content searches, verify that the full text index is enabled for the scan.
   Check whether an UltraSearch exclude filter is active.

2. Check the logs on the DataCentral server
   Check the Windows Event Log for errors from SpaceObServer Agent and SpaceObServer DataCentral.
   If a file should be searchable but is not, the file might not have been indexed because of an indexing problem shown in the Event Log.

The speed of DataCentral depends on many factors.

The speed of the indexing of documents depends on:

• the number of actually indexable files (xlsx and txt are indexable, but exe not)
• the size of the files
• the type of the files (txt files are usually faster than xlsx/pdf)
• the hardware (RAM and CPU, incl. the number of cores)
• the index location (avoid HDD drives or network storage, prefer an attached SSD drive)

For a small drive, DataCentral can be done within minutes, for larger ones, it can take hours and for very large drives with multiple TB of indexable files (exe files, images, ... don't count), it can even take days. Smaller scans lead to smaller indexes, which can usually be searched through faster. The initial creation of the index takes the longest. The repetitive updating of the index should be far faster, becaues only changes on the file system need to be applied on the index then.

SpaceObServer shows a progress and an estimation for the remaining time of the indexing in the "Configure scans" window in the "State" column of the scan.

The performance of a search is usually faster than without DataCentral and depends on:

• the number of the files found by the search
• the search term and its frequency in the index
• the network between the UltraSearch client and the DataCentral server

We aim to execute most of the searches within a few seconds.