09.2 Docs

Golem's Docs

In order for Golem to work it needs to create a Docker VM, this allows for computations to be completed in an isolated environement from the Requestor and Provider host machines. Below you will see a list of common Docker Errors from Golem.log files, along with some troubleshooting tips.

If you cannot find your golem.log file, check here for instructions on log locations.

Errors with Hyper-V

Symptoms

Environment: Windows, using Hyper-V (not Docker Toolbox)

When adding a task this error message pops up:

Logs: 2019-01-04 17:37:07 CRITICAL twisted Unhandled error in Deferred: 2019-01-04 17:37:07 CRITICAL twisted
Exception: 500 Server Error: Internal Server Error ("error while mounting volume '': VolumeDriver.Mount: exit status 255")

Possible causes

1. SMB Port unreachable

Diagnosis

This should be checked earlier when Golem is starting and reported in logs:

ERROR [golem.docker.hypervisor.hyperv ] Port 445 unreachable. Please check firewall settings.

To double check, make sure the Golem VM is running and type the following command in PowerShell:

docker-machine.exe --native-ssh ssh golem "if nc -z -w 1 $env:COMPUTERNAME 445 ; then echo OK ; else echo Error ; fi"

If the port is reachable the command output will say OK, otherwise it will say Error.
If you're using cmd instead of PowerShell just replace $env:COMPUTERNAME with %COMPUTERNAME% in the command.

Solution

If you're using only the default Windows firewall (Windows Defender):

  1. Go to Windows Defender settings in control panel.
  2. Open "Advanced settings".
  3. Click "Incoming rules".
  4. Find "Golem SMB" rule on the list.
    • If it's not there => open PowerShell as administrator and run the following command:

New-NetFirewallRule -DisplayName "Golem SMB" -Name "GOLEM-SMB" `

-Direction Inbound -LocalPort 445 -Protocol TCP `

-RemoteAddress 172.16.0.0/12 -LocalAddress 172.16.0.0/12 `

-Program System -Action Allow

If it's there => check if it's enabled. If not just click "Enable" in the actions menu on the right.

  1. You should be done! ✅

If you're using custom, third-party firewall: we won't help you. Find out how to open ports in your firewall's documentation. Port 445 should be open for incoming TCP connections from 172.16.0.0/12 subnetwork (don't worry, this is just virtual subnetwork for VMs, you're not opening your port to the world).


2. golem-docker user not present, or improperly configured

Open user management (lusrmgr.msc in terminal) and make sure there is a user named golem-docker.
Then right-click the user name, choose "Properties" and make sure they look like this:

If the "Password never expires" checkbox isn't selected, check it and save. That should be it. ✅


3. Golem's "ComputerRes" directory not shared

Run Get-SmbShare in PowerShell. In the output there should be entries with Golem data directories:

C37A161EDD52B4F2C7C59E6144A47595 c:\users\golem\appdata\local\golem\golem\default\rinkeby\computerres DDE5B5B8924D95D303468DB8048F07A9 c:\users\golem\appdata\local\golem\golem\default\mainnet\computerres

Note: If you're not using the default Golem data directory paths could be different.
If you don't see anything like this in the output that means shares need to be created. Golem should do this automatically on startup. Just click "Yes" to allow it. ✅

⚠️ In case the above dialog box doesn't appear, or despite clicking "Yes" shares are not created, please report to the Golem team for help.


4. Wrong permissions

  1. Go to your Golem data directory.
  2. Right-click "ComputerRes" folder and select "Properties".
  3. Go to "Security" tab and click "Advanced" button.
  4. Open "Permissions" tab and make sure "golem-docker" user has full control for subfolders and files:
  5. Open "Share" tab and make sure "golem-docker" user has full control:
  6. You're done. ✅

5. Other causes

We are aware that there are more possible causes of mount errors than the ones listed above. However, they were not yet properly analyzed and mitigated. To obtain more diagnostic information ("exit status 255" isn't very informative, is it?) you can look into VM's kernel logs. In order to do that run the following command in terminal right after the error occurs (before just make sure that the VM is still running):

docker-machine --native-ssh ssh golem dmesg


Errors with Docker Toolbox

Error Creating Docker VM

ERROR golem.docker.manager b"VirtualBox: error creating VM 'golem': Command '['docker-machine', 'create', '--driver', 'virtualbox', 'golem']' returned non-zero exit status 3."

WARNING golem.docker.manager b'DockerMachine: failed to env the VM: Command '['docker-machine', 'env', '--shell', 'cmd', 'golem']'

If you see either of these two errors in your golem.log file, then follow these troubleshooting steps:

  • Shutdown the Golem App

  • Open Docker Quickstart Terminal (You can find it in your programs or as an icon on your desktop)

  • Run the following commands in Docker Quickstart Terminal one at a time:

docker-machine ls

This command shows if docker is running any errors. It is good to run this command before and after running any of the following docker commands to see if the error is solved.

docker-machine restart golem

This will restart docker’s golem instance.

docker-machine rm golem

This will remove the docker container.

Outside of the Docker Quickstart Terminal you can remove .docker C:\Users\%USER%.docker\machine\machines\golem

You can search for this directory in your Windows search bar and remove it. This will force remove the docker machine from the docker directory. Then run the following command in Docker Quickstart Terminal:

docker-machine create golem --driver virtualbox

This will create and engage the necessary drivers for your docker machine

  • Now you can restart Golem and test.

Docker Directory Error

_WARNING golem.docker.taskthread Task stderr:/usr/bin/python: can't open file '/golem/work/job.py': [Errno 2] No such file or directory

If you receive this error in your golem.log file, then the directory for the docker and data drive is not using the same hard drive as your Golem app. You just need to move your docker directory to the same hard drive as your Golem directory.

Further Troubleshooting

If you run the docker recovery commands and still recieve errors, you can rerun the commands with the debug flag to get more information.

The first one is docker-machine -D ls this will show your active machines and errors if there are any, specific errors can be googled or searched in docker's github for more troubleshooting options. You can also send your debug return errors to chat.golem.network or contact@golem.network

If docker-machine ls shows normal you can try re-creating the machine. Make sure golem is off during these commands.

docker-machine -D rm golem

run Golem

After running these commands if any errors show in the debug return, then you can search for the specific error or contact us for further troubleshooting options. If you receive no errors, you can run docker-machine ls again, then if no errors show you can try to run golem again.

How to make sure Golem VM is running

There are numerous ways to do this. The easiest way to check if VM is ready is to run the following command in terminal:

docker-machine ls

There should be golem machine listed in the command's output with state Running:

If it's not running use this command to start it:

docker-machine --native-ssh start golem

If you continue to see errors, then the debug flag (-D) should reveal the problem.