Deploying a static website efficiently on Google Cloud Storage

I have several static websites hosted on Google Cloud Storage for which the only thing missing is the ability to have a custom domain using SSL.

In my deploying process from the local file system I was using the gsutil cp command that enables gzipping the files by extension (html,css,js). However each time every file was copied which was a pain when I was using several unmodified images.

My first experience with gsutil rsync was bad:

  • I could not gzip the files I needed
  • Just touching a file was causing it to be re-deployed
However I was able to find an optimized way for deployment
  • Split my local folder into 2 folders with the same hierarchy, one containing the content to be gzip (html,css,js...), the other the other files
  • Gzip each file in my gzip folder (in place)
  • Call gsutil rsync in for each folder to the same gs destination
Of course, it is only a one way synchronization and deleted local files are not deleted remotely
For the gzip folder the command is

gsutil -m -h Content-Encoding:gzip rsync -c -r src/gzip gs://dst

forcing the content encoding to be gzippped

For the other folder the command is

gsutil -m rsync -c -r src/none gs://dst

the -m option is used for parallel optimization. The -c option is needed to force using checksum validation (http://stackoverflow.com/questions/39016621/why-is-gsutil-rsync-re-downloading-all-our-files/39996032#39996032). the -r option is used for recursivity.

To do all this, I wrote a script (in dart of course !) that you can try

# Install the tool
pub global activate -s git git://github.com/tekartik/tekartik_deploy.dart

Create a simple web project

# Build
pub build
# Deploy Install my build folder to my bucket
gswebdeploy build/web gs://my_bucket/my_folder

You should see the following

Building synchronization state...
Starting synchronization
Copying file://build/gs/web/gzip/index.html [Content-Type=text/html]...
Copying file://build/gs/web/gzip/main.dart.js [Content-Type=application/javascript]...
Copying file://build/gs/web/gzip/packages/browser/interop.js [Content-Type=application/javascript]...
Copying file://build/gs/web/gzip/packages/browser/dart.js [Content-Type=application/javascript]...
Copying file://build/gs/web/gzip/styles.css [Content-Type=text/css]...          
- [5/5 files][ 12.5 KiB/ 12.5 KiB] 100% Done                                    
Operation completed over 5 objects/12.5 KiB.  

If you run again, you'll see that only modified files are uploaded

Popular posts from this blog

Soving AccessDenied on Google Cloud Storage domain mapping

As I was facing this again trying to create amp pages for my main site www.tekartik.com , here is a concrete example for my changes Adding the proper DNS redirection in my domain provider (OVH in my case) www.tekartik.com. CNAME c.storage.googleapis.com. This was first giving the error: <Error>   <Code>AccessDenied</Code>   <Message>Access denied.</Message>   <Details>Anonymous callers do not have storage.objects.list access to bucket www.tekartik.com.</Details> </Error> the solution was to configure my bucket as a website configuration : gsutil web set -m index.html -e 404.html gs://www.tekartik.com And to make all files I add public by default and not worry about ACL anymore, I can use gsutil defacl ch -u AllUsers:R gs://www.tekartik.com To make previous imported files public, I can use gsutil -m acl -r set public-read gs://www.tekartik.com
Read more

Fixing "DerInputStream.getLength(): lengthTag=109, too big" in Android Studio

This happened to me in Android Studio after AndroidX migration and using the new testing framework. Even deleting the existing ~/.android/debug.keystore was failing for me The solution was regenerate it manually (accept all questions as empty and say yes at the last one) $ keytool -genkey -v -keystore debug.keystore -storepass android -alias androiddebugkey -keypass android -keyalg RSA -keysize 2048 -validity 10000 And copy it $ rm ~/.android/debug.keystore $ cp debug.keystore ~/.android/debug.keystore
Read more

Simple Samba setup on Lubuntu

I have a simple lubuntu server where I setup a ssh server. While it is sufficient to access a driver/folder from another linux machine using sftp, it is a pain from Windows & Mac, so unfortunately, samba is still needed here. For simplicity I needed only one samba user that I could map to a single user on my lubuntu machine Following the various tutorial I ended up doing this with simple commands Install samba $ sudo apt-get install samba Add myuser to the samba $ sudo smbpasswd -a myuser Export the drive I needed (no guest user) $ sudo vi /etc/samba/smb.conf to add the following export (no real option needed here) [files] path = /media/music/files writeable = yes save and Restart samba $ sudo service smbd restart
Read more