GPE/GSE inconsistency due to GSE reuse deleted vids at end of segments

Problem

This issue may present as data loading failing to work as expected. For example, the user may report trying to load 10 vertices and none of them being retrievable afterwards. This issue may also appear as GPE crashes during query execution. == Error message

Error message will differ depending on symptoms.

Diagnosis

This API endpoint can be used to check if the number of deleted vertices matches between the GSE and GPE. The json response will contain a “message” field. If the message is “check failed” then the issue is present. Otherwise, the message will be “check passed”.

curl -X GET "http://localhost:9000/deleted_vertex_check?threadnum=10&verbose=1" > deleted_vertex_check.json

Additional utilities for diagnosing this include the API endpoint for getting a vid from primary ID and querying the primary ID in the GPE. Note that the GPE endpoint is publicly documented, while the idservice endpoint is internal and not documented. Check vid in GSE:

curl 'localhost:9000/idservice?id=SOME_PRIMARY_ID&type=SOME_VERTEX_TYPE'

Check vertex in GPE:

curl -v 'http://localhost:9000/graph/gsql_demo/vertices/SOME_VERTEX_TYPE/SOME_PRIMARY_ID'

Workaround

Dropping all data will workaround the issue. After doing so it is recommended that the user upgrades before reloading.

Solution

First upgrade to version 3.5.1 or later. Then use the following API endpoint. The config change is necessary to allow the endpoint to delete invalid vertices.

(# Append DeleteByForce environment variable to GSE env vars with value of 1)
gadmin config set GSE.BasicConfig.Env "$(gadmin config get GSE.BasicConfig.Env); DeleteByForce=1"
gadmin config apply -y
gadmin restart gse -y
curl 'http://localhost:9000/deleted_vertex_check?threadnum=10&verbose=1&fixGSE=true&fixGPE=true'