Data storage and distribution

The problem

Data storage is not as expensive as it used to be. However, storing large amounts of data in a secure and reliable way still has a sustainable price tag. When we look at cloud services that store petabytes of data you can say that this is still a costly operation.

3D models use far more data storage then an average webpage. For all models the geometry, appearance, scene and animation information must be stored. Depending on the file format this can be a few hundreds of Kilobytes in a compressed format for a simple model to tens of Megabytes for more complex structures and higher detailed models. It is advised to optimize models for web distribution so the amount of data will be minimized. This will not only save on storage capacity but the data also has to travel through the network to the client that will interact with the models in their browser.

Storing the data for a single Cosmos planet or VR application containing multiple models is still manageable and can be done on a single server. Like running a game on your own computer the average available hardware has no problems storing the models needed, even for larger applications. With hard disks of some Terabytes the data for multiple games or planets can be stored easily. But a single server hosting the data creates a risk for failure. When this server goes offline the application is offline as well.

If you run a website on a dedicated server, a virtual private server or cloud server it has at least tens of Gigabytes of storage available, even when using fast SSD’s. This is often far more than needed and most of the available storage remains unused. When developing a 3D application for the web more storage is used but still not the amount available on the average hosting. By utilizing the available storage as cache for other applications, the whole system will benefit from the available resources. It makes the data available on more locations and by storing it multiple times it serves as a backup system as well.

The purpose of a private server or node

The data for the online applications must be stored somewhere on the internet. This will be one of the tasks of a private server. It stores the data similar to how a web server for a website would. Besides storing the data from the owners’ application it will also use the available storage to cache data from other applications on the platform. This will utilize the available storage in the network better than web servers currently do now. A lot of storage space is unused because web servers are often a dedicated host for a single application or website. The allocated storage for the host can not be used by anything else.

Servers or nodes will distribute the data to each other as seeders using a bittorrent protocol. This will duplicate the data on multiple nodes to make it redundant and available even when the original host is not online. But it will also make the system faster because more nodes can serve the data to the clients. Because not all data can be stored on a single server it will only store data that is used by clients in the neighbourhood like a caching system.

When a client demands data for a particular model it uses a magnet link to start downloading the data. The server that receives the request for a piece of data will immediately look for the data in the torrent network and start downloading it as well. All nodes that have the data available will provide it and by requesting the data, it will be duplicated on a new node.

Clients can obtain the data needed for landparts and surrounding land or a planet from any server but will receive it from the fastest available in their neighbourhood. When there is more need for a particular piece of data, because more users are on the same location or planet, more nodes will have it in cache and as a result it becomes more distributed and faster available to the clients.

Another advantage of this torrent mesh network is that failure of a particular node will not affect the system. Since all the data is available on multiple servers it can be delivered from other nodes and as all servers can perform the same tasks it doesn’t matter which one is actually doing the tasks.

Besides storing and distributing data, the server nodes will also perform other tasks (discussed in another post on the Cosmos platform architecture). This makes the whole platform distributed and redundant. If more people participate in the network, it will perform better and it will become more reliable.

The advantage of hosting your own data

Having a private server node online ensures the data of the owners landpart and planet to be available in the network but it will also participate in the performance of the platform. The private server primarily hosts the data for the wallet owner that is running the node, but it also caches data that is used by clients nearby.

Hosting your own data is not mandatory

So, running a server node to participate in the system gives some advantages and will be the backbone for the whole platform. However, to deploy your application it is not mandatory to have your own server. You can create and deploy your creations to the network from your own computer and the data will be cached in the network as well. It is however no guarantee that the data stays in cache when there are no users requiring the data.

Accessing the data

To access data for models in the platform within the browser you need to know what parts of data you need and where to find it. This information is stored in a blockchain. Your location in the virtual world determines the landpart you are on. Via this landpart token, all tokens of the models can be retrieved. The model tokens store the data location as magnet links. With the magnet links the data can be downloaded from the torrent network.

Summary

The storage requirements for the 3D virtual world are higher than traditional websites. Storing large amounts of data in a secure way is still expensive. The solution is to distribute the storage in smaller amounts over multiple nodes.

To make the storage of the data distributed, redundant and widely available we rely on the use of private servers (nodes). The servers will be seeders of data in a torrent network that will store the data of the owner but also cache data needed by clients nearby. By caching the data, it will be duplicated and so it will serve as a backup system as well. It also becomes available from multiple nodes so it will improve the performance of the platform.

When a client wants to access data, the nearby servers will also download the data and therefore duplicate it. By duplicating the data it will also become available on more nodes. This will immediately speed up the download process.

Cosmos platform architecture

In this article we describe the Cosmos Cyber Reality platform from a technical perspective. We will explain how we are going to realize a true distributed and decentralized system (read about the differences between distributed and decentralized here). The user interface will be available for everyone in the browser, but behind that there are networks and different techniques to provide the data and the security for the platform. By combining available technologies we will achieve true distribution of tasks throughout the whole system.

As long as the system is under development, details in the design of the system may be changed due to performance issues or new available solutions to existing problems. Some problems will only become visible when running in a production environment, but will require redesigning parts of the system. The high level design will remain the same, while implementation can be altered to the best possible solution.

Three layered system

The platform can be divided into three layers:

  • the client layer
  • the blockchain layer
  • the server layer

The client layer

The client layer is the part of the system that runs in the user’s web browser. It renders the user interface with 3D models using web standards and makes it available in all modern browsers that support WebVR. This can be on a PC/Mac, tablet, mobile phone or VR headset. With a VR headset or a mobile phone in a VR goggle you can experience the metaverse in virtual reality. On a monitor or without the use of a VR goggle, the interface will be rendered in a three dimensional display.

The software runs completely in the client browser. This means that, once the software is loaded in the browser, there is no dependency of a single web server. The blockchain network and torrent mesh will provide the data needed in the user environment.

The client interface will be provided from the web portal on cosmoscr.io, but you can also download it from the repository and open it from the local drive. This makes it independent from a single server or website. However, the client will still need an internet connection because all the models and other data are only available online. An example for this can be seen in the MyEtherWallet software. The interface is provided on the website https://www.myetherwallet.com/, but you can also download the code from the github repository and start it without using the website, or host it on your own website. It uses data from the Ethereum blockchain to show in the browser.

The blockchain layer

The blockchain layer fulfills multiple use cases in the platform. It will be the ledger for the payment system in the platform, but it also keeps ownership data for the land parcels and other assets in the system. Using smart contracts some logic or automated tasks, like entrance payment or payed model usage, can be programmed in this layer.

The blockchain provides an open and trustful decentralized data structure for the state of the platform. This can be utilized in multiple ways.

First of all it stores the distribution of the $CYBR tokens to the wallets used in the system. It then validates all transactions of this ERC20 standard fungible token when payments are made anywhere on the platform. This provides a uniform way to make payments in the Cosmos. Whether it is to pay for land, models or other ownable assets, pay for entrance to planets with unique experiences or to buy real goods in a shop, it can all be done with the same digital currency.

Second use is the management of unique assets in the platform using a ERC721 or equivalent non-fungible token (NFT). This token can hold unique assets that can be owned and traded in the platform. It will be used for land parcels (see the article about Cosmos virtual land) and for models. The land tokens will be stored with the location on the mainland and are owned by a wallet holder. This wallet holder can then add 3D models on that land, so a unique visible building will appear on the mainland. For all the models on the land parcel a token will be stored in the blockchain containing the link to the data that is needed to render the objects. This link will be a magnet link to a resource stored in the torrent network provided by the server layer of the platform. In some cases, where the data is using a payment model, it will also store the smart contract address to unlock the data.

Third use is related to the smart contracts. Smart contracts are coded programs that can be executed on the blockchain. This can automate tasks or execute checks on demand to verify access or usage of models in applications. Some standard contracts will be made available on the platform but users can also create their own contracts containing the logic of their specific needs.

The server layer

The server layer is the last separated part of software that is used in the platform. It will provide a distributed data backbone for the system. It will also be used to perform interaction tasks for the clients and can be used in the governance system for the platform.

The private servers will primarily be used to store and distribute data for the network. This is an important task because there will not be a central system to store all the data for the platform. A centralized solution must be prevented for multiple reasons, the most important are:

  • No single authority controlling the data
  • Storage of large amounts of data is expensive
  • Risk of failure of the whole system
  • Prevention of unauthorized modification of data

To provide and distribute the data, servers will use a BitTorrent network. Using the BitTorrent protocol the servers will create a mesh network where the data is never all in one location but distributed in parts over multiple machines and it will create redundancy of the data. The client can download the required data from multiple nodes at the same time. This will increase the performance for accessing the data but also provides a system that is not dependent on a single server. When more people participate with private servers, the system will become more stable.

Besides the data distribution task of the private servers they will be used to increase the performance of the whole system. This is done by providing some data structures that are stored in the blockchain directly to the clients and validating transactions on that data, acting as a light node in the Ethereum network. It will be like a caching system for the database in the blockchain that can be accessed quickly and reliable. It will only be used for data that is not changing to often like the land grid. The server will update the data when changes take place in the blockchain data, but can deliver it to multiple clients afterwards.

Then there is the interaction between clients. To prevent direct access from client to client, which is an unwanted situation because hackers can target clients, the servers will act as middleman. Communication will be end to end encrypted, but will always be directed through one or more server nodes. This way, clients can not get a hold of each other’s IP address.

Now why would anyone run a private server?

We rely on the use of private servers to create a true distributed system. By participating you help keeping the system distributed and reliable. But there is also some personal gain.

When you start your server and link your public wallet address, the server will collect your properties and secures the data for you. This way it will guarantee the availability of the data on the network when your server is online. The torrent system of the server will eventually flush data when its storage limit is reached and new data is requested. By connecting your wallet this will not happen for the data you own. You can run multiple servers holding your data to create redundancy in a private setup.

The costs for running a private server are limited. The only requirements to run a server will be an internet connection and some data storage available. In case you run a planet that can make you money (because of entrance fees or payments in a shop or casino) it will be profitable to run your own server to ensure the availability of your application data. When hosting large applications that can make big amounts of money, you will make sure your servers are securing their own redundancy.

How do you run a server?

The server software will be provided as a Docker image. This is a ready to run virtual microservice system that can run on any machine and on any operating system supported by Docker. For more information about Docker please visit https://www.docker.com.

To run a Docker you can use cloud systems that support Docker containers or you can run it on your own system, even your PC. Just install Docker and use the start command for the Cosmos private server image. It will run in the background, performing the tasks it was programmed to do.

Open source software

To keep the platform open, transparent and available for improvement, all software that is used for the platform will be open sourced and publicly available for everyone. This includes the client software running in the browser, the server software running in Docker containers and the smart contracts for the blockchain platform.

The repository will be maintained by the Cosmos CR team but everyone can help building the system and making it better. Guidelines for software development and testing will be presented in the repositories.

We encourage people to create open source software on the platform. However we will not demand it for applications created for the platform. We think that usage and creativity must be rewarded, so all applications and models are allowed to be published in the Cosmos Cyber Reality metaverse. Everyone who wants to build and profit must be able to do so!

The foundation will also reward developers and designers for their contributions to the platform. General purpose tasks can be assigned to the community in exchange for $CYBR tokens. It can be seen as contracted work in favor of the foundation and the whole community.

The Virtual World of Cosmos

In this article we would like to explain more about the virtual land in Cosmos Cyber Reality. The virtual land is one of the most important parts in making the decentralized virtual world possible. That is why it needs to be flawless. The virtual world should be vibrant and exciting and full of use cases. All of this in a complete VR experience.

The Cosmos Universe

Cosmos is a virtual reality world that is made out of many land parcels. A land parcel is your own piece of the universe! You can buy it and hold it as an NFT (non-fungible token. This gives the token holder true ownership of the land parcel.
(Please read more about what an NFT is here ).

A land parcel can be compared to your own domain name on the internet, and allows you to build your own 3D environment or application on the Cosmos platform. Great examples are games, web shops, VR showrooms and much more. The only limit is your imagination. Don’t worry if you don’t have any programming experience, you can buy your own virtual real estate in the community store and enjoy the benefits the virtual world has to offer.

The cosmos universe starts with a total of 50000 land parcels that are created in the first land forming. The 50000 land titles will form the first city and this will slowly expand in the future. Starting small and controlled expanding will keep virtual land scarce, but open for new people to join and for new use cases to emerge.

There will be two types of parcels:

Public land

Public land will be used by the non-profit foundation to create the infrastructure. Examples are roads, stations, parks, water and everything needed to make the city attractive. But the foundation will also build some showcases to give an impression of the possibilities.

Community land

The community land consists of the land parcels sold to the community and the NFT gives them true ownership of their virtual land property.

Every land parcel has the same size of 10m x 10m (approximately 33ft x 33ft) and is adjacent to another land parcel. All parcels have x and y coordinates and these determine the unique locations of the parcels.

The first city will be generated in a circle with a diameter of 262 land parcels. In the center there will be a central square with foundation buildings.

Land distribution

A maximum of 15000 land parcels will be given away during the ICO phase (one land parcel for every 5 eth). 15000 land parcels will be used as public land and will be owned by the non-profit foundation. The other 20000 land parcels will be sold to the community in an auction.

After a year new land parcels will be generated in a controlled way to allow new use cases to emerge while keeping the land scarce. This land will be sold by the non-profit foundation and the tokens raised will 100% go back into the ecosystem of the virtual world. The new land parcels can be added to the existing mainland at the location of choice.

Planets

Land parcels are limited in space and we did that for a reason. The mainland should serve as a place where people can walk from use case to use case without having to walk miles. The mainland should be compact with a lot to do. For some applications, for instance a web shop or a showroom, the space of a land parcel will be enough, but for games people would probably need more space.

That’s why we created planets. Planets are an extension to your land parcel and allow you to create bigger environments suitable for theme parks, games and much more. To buy a planet you will always need a land parcel in the mainland that serves as portal to your planet.

Every land parcel gives you the right to own one planet. The planet is also a non-fungible token (NFT), but instead of owning the non-fungible token your mainland parcel owns the planet. So indirectly you control the planet, but you can never sell it separate from your land parcel. Your mainland and planet are connected and always act as one. It is possible however to connect more mainland parcels to one planet. This creates more portals to your planet at different locations on the mainland.

Dynamic land system

One of the features that will make Cosmos CR unique is the dynamic land system. As explained before, the land should be vibrant and exciting. The virtual land won’t be really popular if there are empty parcels everywhere as that would make the city look chaotic. People losing their private key can be a real issue, especially if this person owns more than one land parcel. The virtual world will also need some rules so that it will be a great place for all ages.

This is why we’re working with a dynamic land system. Everyone will have their own location and they will keep this location as long as they can prove they still own their private key. People will need to do a small task now and then (the frequency is to be determined) to prove ownership of their land. If land owners fail to prove this several times they will be removed from their location and their land parcel will be non-active. The land owners will not lose their land parcel however but only their location. If they want to, they can reconnect their land to the virtual world on an available location.

Land owners will never lose their location if they perform the needed actions, and land owners can decide to change location to any available parcel or connect it to the outside of the mainland. This is great if you see a great available land parcel that suits your use case or if you decide to change your land to a use case that needs to be in a specific district because of the content.

On-chain governance

The cosmos will have some rules to keep the land fun and appropriate for everyone. Land owners that don’t follow the rules should be warned and after a couple of warnings even be removed from their location (they will not lose their land). But this should not be done by a single entity or organization as this would create a centralized weakness in the system that will not work for a decentralized metaverse. For this we’re looking to create an on-chain governance to prevent people from crossing the boundaries. This doesn’t mean we do not support all kinds of content, because everything should be possible, but to allow areas fit for all visitors. That’s why for some specific applications and environments we will make specific districts that can also grow with the universe. An example can be an adult district.

Adult district

Adult content can be a part in the future of VR. For instance gambling and 18+ use cases.
The experiences are much more realistic than ever before, but to separate this from people that might find this offensive we will create an adult district. Here people can do everything they like to be available for adults, from adult shops to webcam services or strip bars.

Going to a casino and to actually be able to talk to the other poker players can make it much more fun and can make the experience much more realistic.

For more questions about the Virtual world and land parcels you’re welcome to join our open communities on different social media platforms. The links can be found on the website https://cosmoscr.io