Channel that uses AWS S3 as backend and uses the AWS SDK for C++ to access S3. More...
#include <S3.h>
Inheritance diagram for FMI::Comm::S3:
Collaboration diagram for FMI::Comm::S3:
Public Member Functions | |
| S3 (std::map< std::string, std::string > params, std::map< std::string, std::string > model_params) | |
| ~S3 () | |
| void | upload_object (channel_data buf, std::string name) override |
| Function to upload data with a given name / key to the server, needs to be implemented by the channels and should never be invoked directly (use upload instead). More... | |
| bool | download_object (channel_data buf, std::string name) override |
| Function to download data with a given name / key from the server, needs to be implemented by the channels. Returns true when download was successful, false when file does not exist. More... | |
| void | delete_object (std::string name) override |
| Delete the object with the given name, needs to be implemented by channels. More... | |
| std::vector< std::string > | get_object_names () override |
| List all the currently existing objects, needs to be implemented by channels. Needed by some collectives that check for the existence of files, but do not care about their content. More... | |
| double | get_latency (Utils::peer_num producer, Utils::peer_num consumer, std::size_t size_in_bytes) override |
| Performance model of an individual channel, provides latency for a simple exchange with a number of producers / consumers. More... | |
| double | get_price (Utils::peer_num producer, Utils::peer_num consumer, std::size_t size_in_bytes) override |
| Cost model of an individual channel, provides price for a simple exchange with a number of producers / consumers. More... | |
 Public Member Functions inherited from FMI::Comm::ClientServer | |
| ClientServer (std::map< std::string, std::string > params) | |
| void | send (channel_data buf, FMI::Utils::peer_num dest) override |
| Constructs file / key name based on sender and recipient and then uploads the data. More... | |
| void | recv (channel_data buf, FMI::Utils::peer_num dest) override |
| Waits until the object with the expected file / key name appears (or a timeout occurs), then downloads it. More... | |
| void | bcast (channel_data buf, FMI::Utils::peer_num root) override |
| Root uploads its data, all other peers download the object. More... | |
| void | barrier () override |
| All peers upload a 1 byte file and wait until num_peers files (associated to this operation based on the file name) exist. More... | |
| void | reduce (channel_data sendbuf, channel_data recvbuf, FMI::Utils::peer_num root, raw_function f) override |
| All peers upload their data. The root peer downloads these objects and applies the function (as soon as objects become available for associative / commutative functions, left-to-right otherwise) More... | |
| void | scan (channel_data sendbuf, channel_data recvbuf, raw_function f) override |
| All peers upload their data and download the needed files to apply the function. Left-to-right evaluation order is enforced for non-commutative / non-associative functions. More... | |
| virtual void | download (channel_data buf, std::string name) |
| Try the download (using download_object) until the object appears or the timeout was reached. More... | |
| virtual void | upload (channel_data buf, std::string name) |
| Uploads objects and keeps track of them. More... | |
| void | finalize () override |
| Deletes all objects that were created during the execution. More... | |
| double | get_operation_latency (Utils::OperationInfo op_info) override |
| Performance model for collective operations, provides the latency for the given operation. More... | |
| double | get_operation_price (Utils::OperationInfo op_info) override |
| Cost model for collective operations, provides the price for the given operation. More... | |
| Public Member Functions inherited from FMI::Comm::Channel | |
| virtual void | gather (channel_data sendbuf, channel_data recvbuf, FMI::Utils::peer_num root) |
| Root gathers data from all peers. More... | |
| virtual void | scatter (channel_data sendbuf, channel_data recvbuf, FMI::Utils::peer_num root) |
| Scatter data from root to all peers. More... | |
| virtual void | allreduce (channel_data sendbuf, channel_data recvbuf, raw_function f) |
| Apply function f to sendbuf of all peers, make result available to everyone. More... | |
| void | set_peer_id (FMI::Utils::peer_num num) |
| Helper utility to set peer id, ID needs to be set before first collective operation. More... | |
| void | set_num_peers (FMI::Utils::peer_num num) |
| Helper utility to set number of peers, needs to be set before first collective operation. More... | |
| void | set_comm_name (std::string communication_name) |
| Helper utility to set the communicator name, should be set before first collective operation to avoid conflicts with empty communicator name. More... | |
Additional Inherited Members | |
| Static Public Member Functions inherited from FMI::Comm::Channel | |
| static std::shared_ptr< Channel > | get_channel (std::string name, std::map< std::string, std::string > params, std::map< std::string, std::string > model_params) |
| Create a new channel with the given config and model params. More... | |
| Protected Attributes inherited from FMI::Comm::ClientServer | |
| std::map< std::string, unsigned int > | num_operations |
| Ensures that there are no file / key name conflicts when a collective operation is used multiple times, values are integrated into the file / key name for these operations. More... | |
| std::vector< std::string > | created_objects |
| Tracks all created objects such that they can be selectively deleted when finalize is called. More... | |
| unsigned int | timeout |
| Backoff time when an object does not exist. More... | |
| unsigned int | max_timeout |
| Maximum time to wait until an object appears, exception is thrown when this value is exceeded. More... | |
| Protected Attributes inherited from FMI::Comm::Channel | |
| FMI::Utils::peer_num | peer_id |
| FMI::Utils::peer_num | num_peers |
| std::string | comm_name |
| Can optionally be used by channels to avoid resource conflicts that may occur because of multiple concurrent communicators. More... | |
Detailed Description
Constructor & Destructor Documentation
◆ S3()
|
explicit |
◆ ~S3()
| FMI::Comm::S3::~S3 | ( | ) |
Member Function Documentation
◆ delete_object()
|
overridevirtual |
Delete the object with the given name, needs to be implemented by channels.
Implements FMI::Comm::ClientServer.
◆ download_object()
|
overridevirtual |
Function to download data with a given name / key from the server, needs to be implemented by the channels. Returns true when download was successful, false when file does not exist.
Implements FMI::Comm::ClientServer.
◆ get_latency()
|
overridevirtual |
Performance model of an individual channel, provides latency for a simple exchange with a number of producers / consumers.
Implements FMI::Comm::Channel.
◆ get_object_names()
|
overridevirtual |
List all the currently existing objects, needs to be implemented by channels. Needed by some collectives that check for the existence of files, but do not care about their content.
Implements FMI::Comm::ClientServer.
◆ get_price()
|
overridevirtual |
Cost model of an individual channel, provides price for a simple exchange with a number of producers / consumers.
Implements FMI::Comm::Channel.
◆ upload_object()
|
overridevirtual |
Function to upload data with a given name / key to the server, needs to be implemented by the channels and should never be invoked directly (use upload instead).
Implements FMI::Comm::ClientServer.
The documentation for this class was generated from the following files: