When I run memsql-report collect, I see that the report is not collected from the same node every time. Why is that?
$ sdb-report collect
✘ Failed to collect report for host dev-memsql-host1.com
✓ Collected report for host dev-memsql-host2.com
Please note that it happened since we migrate from OPS to Toolbox.
The sdb-report collect accepts our high verbosity debug flag -vvv. You should run the cluster report with the debug option memsql-report collect -vvv to get additional information as to why the cluster report is failing to collect on the nodes.
In the verbose output, you may see a message like this:
$ sdb-report collect -vvv
Debug 2020/11/05 14:17:04 collect.go:213 Error: unable to find or setup a memsql-report binary on the host dev-memsql-host1.com
this may be due to not having access to /tmp or $TMPDIR on remote host
try using "--remote-temp-dir" flag
2020/11/05 14:17:04 logging.go:52 Command #141 failed with exit code 126 in 47.453326ms
Unlike MemSQL Ops, with Toolbox, the $TEMPDIR should have executable permissions for memsql.
You need to check, by connecting to the host reported, that /tmp was mounted with the exec option. Most likely on dev-memsql-host1.com, the /tmp was mounted with the noexec option.
You need to remount it with the exec option:
sudo mount -o remount,exec /tmp
More information about the cluster report can be found here.