How to select the most suitable Google Cloud Storage solution?
Google is offering many cloud services and finding the best one which is matching with your requirements can be confusing. Let's just focus on Google Cloud Storage for now. The offered storage solutions by Google are:
Cloud SQL, BigTable, Cloud DataStore, Cloud Storage and Cloud Spanner.
The selection of each solution is based on the multiple criterias such as type of data if they are structured or unstructured, type of the transaction, how much scalability is important and so on. For the guide, I have also included BigQuery.
- If your data is not structured (Such as image or video):
- Go for Cloud storage, it supports Petabytes of data storage. This is a great option for storing the objects which are data with any formats. The objects are stored in Buckets. More details: https://cloud.google.com/storage/docs/introduction
- If your data is structured:
- If the workload is transactional, and you need to have a SQL-based transaction, and one database is enough, go for Cloud SQL. It is a fully managed RDBMS and you can use it with MySql or PostgreSql or Sql Server. It provides data encryption, replication, maintenance and many more. More details: https://cloud.google.com/sql/docs/introduction
- If the workload is transactional and you need to have a SQL-based transaction and need multiple databases or you need horizontal scalability across multiple locations, go for Cloud Spanner. More details: https://cloud.google.com/spanner/docs/quickstart-console
- If the workload is transactional and you need to have a NoSQL-based transaction (which means use keys for searching data), go for Cloud Datastore, it is going to be replaced by Firestore soon. This service is supporting documents and secondary index. More details: https://cloud.google.com/datastore/docs/concepts/overview
- If the workload is analytical, in this case you have two options: BigQuery & Cloud BigTable
- BigQuery is a data warehouse and a query engine for databases which don’t change that much and you may only add some data to it. You use it when you want to find some analytics about your data. More details: https://cloud.google.com/bigquery/docs/introduction
- BigTable is a database which supports Hbase API and key-value access. You can use it for creation of scalable applications, which means you need to read and write the data and scalability is important and you expect low latency. Some good use-cases to use BigTable are when you have a huge amount of time series data (sensors or stock price) or clickstream data or analytic use-cases such as recommendations or personalization. More details: https://cloud.google.com/bigtable/docs/overview
To find out more details please check some of the indicated links of below references.
- https://stackoverflow.com/questions/39919815/whats-the-difference-between-bigquery-and-bigtable
- https://www.coursera.org/learn/gcp-big-data-ml-fundamentals/lecture/s3wa2/approach-move-from-on-premise-to-google-cloud-platform
- https://stackoverflow.com/questions/53489250/price-aside-why-ever-choose-google-cloud-bigtable-over-google-cloud-datastore