Parallelization of RASPA simulations with Golem

[nowakoww]

Dear Professor Dubbeldam, Dear RASPA Users Community,

We are Golem. Golem is a project that aims at building the first truly decentralized supercomputer, creating a decentralized sharing economy of computing power. Golem connects computers in a peer-to-peer network, enabling users (requestors) to rent resources of other users' (providers) machines, from PCs to entire data centers. These resources can be used to complete computationally intensive tasks faster, cheaper and easier. Golem is capable of computing a wide variety of tasks, form CGI rendering, through machine learning to scientific computing. Apart from requestors and providers, an important part of Golem's ecosystem are software developers who use Golem as a channel for distribution  of their software. This software is intended to be used out-of-the-box by requestors and providers to submit and perform computing tasks. An example of such application could be a plug-in for an existing rendering software (like Blender or 3D Max) which enables its users to easily submit rendering of a 3D scene to hundreds or thousands of nodes in the Golem network and get things done much faster. Such a plug-in could hide all the complexity of task parallelization from the users. A Golem application doesn't have to be a plug-in. It could be a standalone application or python library as well. More about Golem you can find at our website: https://golem.network/

We are advanced on our way to make the above vision come true and rendering is the first use case we've implemented to demonstrate real Golem usage. We are now working hard on integrating a variety of new use cases to Golem in cooperation with our partners both from business and academia. We are especially interested in scientific computing, including molecular modelling, and so we came across RASPA which seems to be a good fit for Golem.

As a proof of concept we have implemented a python library that allows RASPA users to perform simulations using task-farming approach in the Golem network. With this library one can write workflows like the one described here (https://github.com/numat/RASPA2/wiki/Workflow). However, we are not experts in molecular modelling, thus we would need some advice from domain experts and RASPA users. We would appreciate if you could  help us with the two points below. Together we could work out a solution that, on one hand, would be useful for RASPA users and, on the other hand, would be another practical Golem use case.

• Please give us some light on the domain and RASPA software itself by answering the following questions:
  
  
  
  
  • What are the most common usage scenarios for RASPA? Is the workflow described here (https://github.com/numat/RASPA2/wiki/Workflow) a typical one or maybe there's no common way of using RASPA and it heavily depends on particular problem at hand?
  
  
  • What is an average size of the input/output data? What are the biggest inputs/outputs that are used in pracitce (or RASPA is capable to deal with)?
  
  
  • What type of data (formats) RASPA produces as an output for different types of calculations/algorithms? Are these just numbers or target molecules that can be then visualised (in iRASPA for example)?
  
  
  • Could you please provide us with some real-world examples of input and resulting output data along with the desccription of workflows that were used to produce those results. This would give us a better understanding of real usage scenarios and would help us to consider best parallelization scenarios using Golem?
  
  
  • Is it possible to run simulations directly from iRASPA or is it intended only for building and visualizing models of molecules and simulations needs to be run separately using RASPA command line interface or from Python scripts?
  
  
  • Are there ways to verify if the RASPA simulation results are valid, e.g. using statistical methods? In Golem, in case a task is computed on an untrusted node, there should be a way to validate the results. It could be done by redundancy (computationally expensive) or using other less expensive methods.
    
    
  
  


• We would also like to encourage you to play around with our Golem python client for RASPA. Basically we did all the heavy lifting so that the user can focus on doing his job. We appreciate trying out and giving us feedback. To help you get started we prepared installation manuals and example RASPA2 code.
  
  Our integration with this particular task is inspired by the workflow I mentioned above (https://github.com/numat/RASPA2/wiki/Workflow).
  
  In general we have two separate computation environments: testnet (rinkeby) and mainnet. RASPA2 integration is available only as a separate branch ('glambda') in our main repository. In other words it has no real adoption yet (in both environments). One has to set up his own workers on local computer to run the examples. We plan to merge it (or not) after receiving feedback.
  
  Installing from source is described here:
  
  https://github.com/golemfactory/golem/wiki/Installation#running-from-the-source
  
  Make sure you have checked out `glambda` branch.
  
  Now that we have Golem installed on our computer it's time to try integrating our simulation glue code with it. We provided python library called 'golemrpc' (along with integration examples) to do so. For setup instructions please see both:
  
  https://github.com/golemfactory/golemrpc
  
  and the examples:
  
  https://github.com/golemfactory/golemrpc/tree/master/golemrpc/examples
  We are most interested in knowledge about user workflows. How can we do better to ease the process for RASPA2 users? Please ignore cumbersome local node's setup for now. It will be solved by design if we decide to move forward with RASPA and allow it on testnet.
  
  More information about the project can be found here:
  
  https://golem.network/documentation/understanding-beta/
  
  We realize that the above installation process of the Golem client might not be very straightforward and we are working on a so called thin client installation of which would be as simple as possible. We will share it with you soon. In the meantime we encourage you to try out the current version.
  
  

Our future plans include also parallelization of RASPA simulations with MPI (e.g. using spatial or force decomposition) as we are currently working on implementation of MPI on Golem.

In case of any questions or problems from you regarding Golem, we will be more than happy to help you.

Best regards,
Wiktor Nowakowski

[nowakoww]

Hello,

It's Golem here. As mentioned in the post above, we have prepared a thin client. It hides all the complexity of Golem from RASPA users as much as possible. There's no need to install Golem on your computer. Now, you only have to install the thin client, provide connectivity details and run the example RASPA calculations provided within the installation pack to test the installation.

The thin client doesn't contain any Golem-specific components. It just sends computing tasks along with input data to a remote server that acts as a Golem requestor node. Currently this requestor node is hosted on AWS and managed by the Golem Team. The requestor node knows how to speak to the Golem network using the Golem protocol in order to find provider nodes which will actually do computational work in parallel. The more provider nodes there are in the Golem network, the better parallelization we can achieve and obtain results faster.

In the installation pack there are two main directories:
\golemrpc - contains thin client code,
\examples - contains example python script raspa_remote.py and \examples\cifs subfolder with input data.

We've tried to ensure that the code is well documented. To get familiar whit the thin client and the example, please read comments in the code. Also, don't hesitate to contact us either on this forum or via email or any other communication channel we use (see https://golem.network/).

How the example works

raspa_remote.py reads input .cif files and instructs remote requestor node to create computation tasks (one task per one input file) in the Golem network which can be then executed by provider nodes. The provided example runs get_helium_void_fraction() for two selected .cif files from \examples\cifs. After completing the calculations the results for each task are saved in \examples\cifs\<task_id>-output folder, e.g. \examples\cifs\40e0d906-20aa-11e9-8e1c-c772b47c74b3-output.

You can, of course, modify the example to perform your own RASPA calculations on your own input data.

The nature of the Golem network enables performing calculations of parallel tasks on virtually unlimited number of nodes (depending on how many independent subtasks your entire computational task can be divided). However, for the purpose of testing the RASPA thin client we have setup a test environment consisting of just several provider nodes. This means that you can perform calculations on several input files in parallel.

How to install the client and run the example?

The installation is very straightforward. Please follow the instructions here: https://github.com/golemfactory/golemrpc

Note that the thin client connects with the remote Golem requestor node on AWS. To enable secure connection you need credential files: 'golemcli_aws.tck' and 'rpc_cert_aws.pem'. To obtain those files, please contact michal.plebanski@golem.network or wiktor.nowakowski@golem.network.



We would much appreciate your feedback about using the thin client. We would also appreciate if you spend a small quantum of time to answear our questions from the post above. We are open to any discussions on any time.

Best regards,
Wiktor