Cluster Expansion Failed

Problem

Cluster expansion was interrupted due to export gsql data failure.

Error message

Error Code:

Export gsql/gui data
Error: ExternalError (Failed to export gsql/gui data; Failed to export gsql data; Failed to query GSQL with error:<nil>, code:500, url:https://xxxxxxx:14240/gsqlserver/gsql/export?path=/app/data/tgraph/expansion-export-GSQL-0)

Diagnosis

There is a known bug in TG 3.2.3 when DataRoot and TmpRoot are located from different disk, which causes the below rename file error shown in gsql log.

Rename file or directory '/app/data/tgraph/expansion-export-GSQL-0' to '/app/temp/tgraph/tmp/gsql/tmp/expansion-export-GSQL-0_1653019588711' failed, will use copy/move instead.
E@20220519 23:06:28.712 tigergraph|localhost:59646|00000000143 (IOUtil.java:197) java.nio.file.DirectoryNotEmptyException: /app/data/tgraph/expansion-export-GSQL-0
java.nio.file.DirectoryNotEmptyException: /app/data/tgraph/expansion-export-GSQL-0
        at java.base/sun.nio.fs.UnixCopyFile.ensureEmptyDir(UnixCopyFile.java:384)
        at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:484)
        at java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267)
        at java.base/java.nio.file.Files.move(Files.java:1422)
        at com.tigergraph.a.b.h(IOUtil.java:193)
        at com.tigergraph.schema.grpc.r.h(ControllerClient.java:443)
        at com.tigergraph.schema.handler.ExportImportHandler.a(ExportImportHandler.java:201)
        at com.tigergraph.schema.handler.ExportImportHandler.a(ExportImportHandler.java:66)
        at com.tigergraph.schema.handler.BaseHandler.handle(BaseHandler.java:298)
        at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77)
        at jdk.httpserver/sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:71)
        at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:80)
        at jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:692)
        at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77)
        at jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:664)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
E@20220519 23:06:28.712 tigergraph|localhost:59646|00000000143 (ControllerClient.java:446) Copy file or move directory '/app/data/tgraph/expansion-export-GSQL-0' failed.

Workaround

If the below solution of upgrade is not possible, contact TigerGraph support for a patch for version 3.2.3.

Solution

The issue has been resolved in TG 3.6. To resolve, please upgrade.