References » Reprocess All Files » Server microservices

In some scenarios, it may be necessary to run the server microservices against all files and not only based on file changes or new files. When this special set of microservices are started, each one will reprocess every relevant file scanned by Yadle. Upon completion, the individual microservices will stop and exit automatically.

Why Reprocess All Files?

It is typically not necessary to invoke this set of containers, however there are cases where it makes sense to do so.

An examples of when it may be necessary to run the server microservices against all files:
  • There has been an update to Yadle’s AI that requires reprocessing of all files.

In this scenario, it is crucial to reprocess every file already scanned by Yadle to have the latest AI model applied. Since the underlying files are not changing in any way, there won’t be any change event to tell the AI microservices to reanalyze and apply tags. Thus, the server microservices need to run on all files to reprocess.

When invoking this set of server microservice containers (running in “allfiles_mode”), the original microservices (running in “changes_mode”) will continue to process new files and changes. Once the microservices running against all files have completed, the containers will stop automatically.

If you have been instructed by someone at Yadle to start this set of microservices, refer to the steps below.

Starting Server Microservices On All Files

Step 1: Start Yadle server microservices in “allfiles_mode”.

Go to server microservices directory and run start_yadle_all script.

$ cd /opt/yadle/microservices/server
$ sudo ./start_yadle_all

Step 2: Check for running Yadle containers.

You should now see the allfiles_mode containers running alongside the other Yadle containers. Confirm this by checking Docker status.

$ sudo docker ps -a

The following Yadle containers should now be running in addition to the standard Yadle containers. Refer to the Starting and Stopping Yadle section to see the list of standard Yadle containers.

NAMES
<org>_allfiles_mode_tags_1
<org>_allfiles_mode_aitags-colossus_1
<org>_allfiles_mode_aitags-krell_1
<org>_allfiles_mode_filesequence-thumbnails_1
<org>_allfiles_mode_filetype-icons_1
<org>_allfiles_mode_imagehash_1

Stopping Server Microservices On All Files

The allfiles_mode containers will exit one by one as they complete the re-processing of every scanned file with Yadle. If it’s necessary to stop the allfiles_mode server microservices before they have completed there is a separate stop_yadle_all script for that. Restarting the allfiles_mode containers again will resume where they each left off. They will not start over from the beginning.

Step 1: Stop Yadle server microservices in “allfiles_mode”.

Go to server microservices directory and run stop_yadle_all script.

$ cd /opt/yadle/microservices/server
$ sudo ./stop_yadle_all

Step 2: Check for no running allfiles_mode containers.

Confirm there are no running allfiles_mode Yadle containers by checking Docker status.

$ sudo docker ps -a