Aws on Miro's World

How klassify.com went live: one static page, GitHub Pages and Route 53

Thu, 03 Sep 2026 15:30:00 +0200

My company, Klassify s.r.o., has had a domain for a long time before it was even registered (I registered klassify.com way back in early 2000 in Canada, but never actually did anything with it) - and a web page since this afternoon. For all that time klassify.com carried only my email - there was no A record at all, nothing was served, and whoever looked up the company from an invoice or a contract got a browser error and, I suppose, some doubts about whether the company exists. So here is how it got fixed in one afternoon, in the order it happened and with the reasoning. Nothing here is clever or sophisticated or the only/the best way. The point is the sequence of creation: there are two places in it where it is quite easy to break something that matters more than a web page.

Reference Architecture as a Code

Mon, 10 Aug 2020 13:00:00 +0800

Many shapes of Architecture
#

According to the wise people, an architecture is the shared understanding that the expert developers/system builders/maintainers have of the system design. It’s purpose is to maximize the probability that the decisions you have to make early in a project are the right ones. It is sort of a blueprint for the future system and captures parts, relations, boundaries, constraints.

History of Faith

Tue, 31 Mar 2020 11:24:48 +0800

The beginnings
#

Once upon a time - actually, in late 2017-early 2018 - there was a QA/DevOps engineer named José who liked containers, Docker and worked for a company named Thinkwrap (that soon would become Pivotree) in Valencia, Spain.

José was responsible (among other things) for setting up integration and QA environments for multiple Hybris projects. Such environment, when being built in traditional way (from physical servers or VMs), normally requires quite a few pieces:

All Day DevOps 2019

Thu, 07 Nov 2019 11:22:48 +0800

All Day DevOps 2019 - Notes
#

I have attended the All Day Devops - https://www.alldaydevops.com/ on 06 Nov 2019 - an event that runs for 24 hours and has multiple tracks of content organized in 4 blocks.

The tracks:

Keynotes
Cultural Change
DevSecOps
SRE
CI/CD
Everything Cloud

The complete list is here: https://www.alldaydevops.com/2019-live-schedule - the play button points to the block video.

AWS Inspection

Thu, 03 Oct 2019 11:22:48 +0800

Bunch of one-liners for AWS
#

.. so that I find them faster next time

aws --output=json ec2 describe-instances | jq -r '.Reservations[].Instances[] | "\n" + .InstanceId + "  :  " + .KeyName + "  =>  " + .PublicIpAddress + " | " + .PublicDnsName,  .Tags[] as $tt | "  ... " + $tt.Key +":"+ $tt.Value'

aws ec2 describe-instances | jq '.Reservations[].Instances[] | .InstanceId + "  :  " + .Placement.AvailabilityZone + "  =>  " + .PublicIpAddress'

aws iam list-group-policies --group-name kops

aws ec2 describe-vpcs | jq '.Vpcs[] | .VpcId + " " + .CidrBlock'

aws iam list-users | jq '.Users[].UserName'

aws --output=json --region=ca-central-1 ec2 describe-vpcs | jq -r '.Vpcs[] | .VpcId + " " + .CidrBlock,.Tags[] as $tt | "  .. " + $tt.Key +":"+$tt.Value'


aws ec2 describe-vpcs | jq -e --raw-output '.Vpcs[] | "\n" + .VpcId + "  :  " + .CidrBlock,  .Tags[] as $tt | "  ... " + $tt.Key +":"+ $tt.Value'

aws ec2 describe-subnets | jq -e --raw-output '.Subnets[] | "\n" + .VpcId + "  :  " + .CidrBlock +" : " + .AvailabilityZone,  .Tags[] as $tt | "  ... " + $tt.Key +":"+ $tt.Value'

aws ec2 authorize-security-group-ingress --group-id sg-bfa45bd4 --protocol tcp --port 31672 --cidr 204.101.219.210/31

How many instances of each type do I have, and in what states?
#

Considering buying reserved instances or thinking about migrating to a newly introduced class?

aws ec2 describe-instances | jq -r "[[.Reservations[].Instances[]|{ state: .State.Name, type: .InstanceType }]|group_by(.state)|.[]|{state: .[0].state, types: [.[].type]|[group_by(.)|.[]|{type: .[0], count: ([.[]]|length)}] }]"

What CIDRs have Ingress Access to which Ports?
#

# This is helpful when you need to perform a survey or audit of your system boundaries. While such a task isn’t ever “easy”, it can go more smoothly with with a summary:

aws ec2 describe-security-groups | jq '[ .SecurityGroups[].IpPermissions[] as $a | { "ports": [($a.FromPort|tostring),($a.ToPort|tostring)]|unique, "cidr": $a.IpRanges[].CidrIp } ] | [group_by(.cidr)[] | { (.[0].cidr): [.[].ports|join("-")]|unique }] | add'

Which Services am I using?
#

aws ce get-cost-and-usage --time-period Start=2019-08-01,End=2019-08-31  --granularity MONTHLY --metrics UsageQuantity --group-by Type=DIMENSION,Key=SERVICE | jq '.ResultsByTime[].Groups[] | select(.Metrics.UsageQuantity.Amount > 0) | .Keys[0]'


# How much they cost
aws ce get-cost-and-usage --time-period Start=2019-08-01,End=2019-08-31  --granularity MONTHLY --metrics USAGE_QUANTITY BLENDED_COST  --group-by Type=DIMENSION,Key=SERVICE | jq '[ .ResultsByTime[].Groups[] | select(.Metrics.BlendedCost.Amount > "0") | { (.Keys[0]): .Metrics.BlendedCost } ] | sort_by(.Amount) | add'

# Instances running

aws ec2 describe-instances | jq -r "[[.Reservations[].Instances[]|{ state: .State.Name, type: .InstanceType }]|group_by(.state)|.[]|{state: .[0].state, types: [.[].type]|[group_by(.)|.[]|{type: .[0], count: ([.[]]|length)}] }]"

CIDR access to ports
#

aws ec2 describe-security-groups | jq '[ .SecurityGroups[].IpPermissions[] as $a | { "ports": [($a.FromPort|tostring),($a.ToPort|tostring)]|unique, "cidr": $a.IpRanges[].CidrIp } ] | [group_by(.cidr)[] | { (.[0].cidr): [.[].ports|join("-")]|unique }] | add'

Lambda runtimes
#

aws lambda list-functions | jq ".Functions | group_by(.Runtime)|[.[]|{ runtime:.[0].Runtime, functions:[.[]|.FunctionName] }
]"

Memory size
#

aws lambda list-functions | jq ".Functions | group_by(.Runtime)|[.[]|{ (.[0].Runtime): [.[]|{ name: .FunctionName, timeout: .Timeout, memory: .MemorySize }] }]"

Lambda Environment variables
#

aws lambda list-functions | jq -r '[.Functions[]|{name: .FunctionName, env: .Environment.Variables}]|.[]|select(.env|length > 0)'

AWS Summit Madrid

Tue, 07 May 2019 11:22:48 +0800

One day round trip Valencia to Madrid and back.

It is 350 km, but AVE (high velocity train) makes it in 1.5 hr. Take that, Via Rail !!

Travelling at 300 km/h is unreal. It is actually quieter and more smooth than Canadian train at 80 - all that makes the speed believable is the velocity display and view out of the window when train goes around something closer to the rails.

AWS Certified Developer - Associate

Sun, 28 Oct 2018 11:22:48 +0800

Got the second certification today

https://www.certmetrics.com/amazon/public/badge.aspx?i=2&t=c&d=2018-11-28&ci=AWS00261816

AWS Certified Solution Architect - Associate

Thu, 07 Jun 2018 11:22:48 +0800

Step 1 done.

https://www.certmetrics.com/amazon/public/badge.aspx?i=1&t=c&d=2018-06-06&ci=AWS00261816&fbclid=IwAR2K8B4FyKWVstBZXCXn5GhYX9iRyB4U2q8l2ROrM_16m-rlclioalnp0O4

And now dilemma: go deeper into AWS or look around what Google and Azure have to offer ?

So many clouds, so little time …

Blog of the day - Tapas Forever

Thu, 07 Jun 2018 11:22:48 +0800

Fellow Canadian describes his path to Spain

https://tapasforever.com/

Creating Kubernetes Cluster in AWS from scratch with kops

Mon, 07 May 2018 11:22:48 +0800

This describes creation of K8s cluster in AWS environment from scratch as done for microservices based eCommerce project.

Pre-requisites
#

AWS account access - IAM with Admin privileges
AWS CLI installed - https://docs.aws.amazon.com/cli/latest/userguide/cli-install-macos.html
CLI credentials (secret key + api Key) - https://docs.aws.amazon.com/cli/latest/reference/iam/index.html
Kubectl and Kops local install ** see https://github.com/kubernetes/kops and https://kubernetes.io/docs/tasks/tools/install-kubectl/

Test of access
#

➜  etc git:(feature/kubernetes) aws --version
aws-cli/1.11.180 Python/3.6.4 Darwin/17.5.0 botocore/1.7.38
 
➜  etc git:(feature/kubernetes) kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.1", GitCommit:"d4ab47518836c750f9949b9e0d387f20fb92260b", GitTreeState:"clean", BuildDate:"2018-04-13T22:27:55Z", GoVersion:"go1.9.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"clean", BuildDate:"2018-02-07T11:55:20Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
 
➜  etc git:(feature/kubernetes) kops version
Version 1.9.0
 
➜  etc git:(feature/kubernetes) aws --profile twfulfill-miro iam list-users | jq '.Users[].UserName'
.. DELETED ...
"twfulfillment.prod.miro.adamy"
....

Set the environment variables
#

These will be used by subsequent kops / aws commands

Install AWS CLI on Mac

Tue, 14 Nov 2017 14:22:48 +0800

Install AWS CLI on Mac
#

The Mac comes with Python 2.7 and no pip

This guide recommends upgrade to Python 3 which I want to avoid: http://docs.aws.amazon.com/cli/latest/userguide/cli-install-macos.html

Unsuccesfull path - keep the Python 2.7
#

➜  ~ pip --version
zsh: command not found: pip

➜  ~ sudo easy_install pip
Password:
Searching for pip
Reading https://pypi.python.org/simple/pip/
Best match: pip 9.0.1
Downloading https://pypi.python.org/packages/11/b6/abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/pip-9.0.1.tar.gz#md5=35f01da33009719497f01a4ba69d63c9
Processing pip-9.0.1.tar.gz
Writing /tmp/easy_install-QnLLJp/pip-9.0.1/setup.cfg
Running pip-9.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-QnLLJp/pip-9.0.1/egg-dist-tmp-03hMh7
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.mailmap'
warning: no previously-included files found matching '.travis.yml'
warning: no previously-included files found matching '.landscape.yml'
warning: no previously-included files found matching 'pip/_vendor/Makefile'
warning: no previously-included files found matching 'tox.ini'
warning: no previously-included files found matching 'dev-requirements.txt'
warning: no previously-included files found matching 'appveyor.yml'
no previously-included directories found matching '.github'
no previously-included directories found matching '.travis'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'contrib'
no previously-included directories found matching 'tasks'
no previously-included directories found matching 'tests'
creating /Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg
Extracting pip-9.0.1-py2.7.egg to /Library/Python/2.7/site-packages
Adding pip 9.0.1 to easy-install.pth file
Installing pip script to /usr/local/bin
Installing pip2.7 script to /usr/local/bin
Installing pip2 script to /usr/local/bin

Installed /Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

➜  ~ pip --version
pip 9.0.1 from /Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg (python 2.7)

➜  ~ pip install awscli --upgrade --user
Collecting awscli
  Downloading awscli-1.11.185-py2.py3-none-any.whl (1.2MB)
    100% |████████████████████████████████| 1.2MB 1.1MB/s
Collecting colorama<=0.3.7,>=0.2.5 (from awscli)
  Downloading colorama-0.3.7-py2.py3-none-any.whl
Collecting s3transfer<0.2.0,>=0.1.9 (from awscli)
  Downloading s3transfer-0.1.11-py2.py3-none-any.whl (54kB)
    100% |████████████████████████████████| 61kB 6.2MB/s
Collecting botocore==1.7.43 (from awscli)
  Downloading botocore-1.7.43-py2.py3-none-any.whl (3.7MB)
    100% |████████████████████████████████| 3.7MB 363kB/s
Collecting docutils>=0.10 (from awscli)
  Downloading docutils-0.14-py2-none-any.whl (543kB)
    100% |████████████████████████████████| 552kB 2.4MB/s
Collecting rsa<=3.5.0,>=3.1.2 (from awscli)
  Downloading rsa-3.4.2-py2.py3-none-any.whl (46kB)
    100% |████████████████████████████████| 51kB 5.9MB/s
Collecting PyYAML<=3.12,>=3.10 (from awscli)
  Downloading PyYAML-3.12.tar.gz (253kB)
    100% |████████████████████████████████| 256kB 3.5MB/s
Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" (from s3transfer<0.2.0,>=0.1.9->awscli)
  Downloading futures-3.1.1-py2-none-any.whl
Collecting jmespath<1.0.0,>=0.7.1 (from botocore==1.7.43->awscli)
  Downloading jmespath-0.9.3-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.7.43->awscli)
  Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194kB)
    100% |████████████████████████████████| 194kB 4.5MB/s
Collecting pyasn1>=0.1.3 (from rsa<=3.5.0,>=3.1.2->awscli)
  Downloading pyasn1-0.3.7-py2.py3-none-any.whl (63kB)
    100% |████████████████████████████████| 71kB 7.8MB/s
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.7.43->awscli)
  Downloading six-1.11.0-py2.py3-none-any.whl
Installing collected packages: colorama, futures, jmespath, docutils, six, python-dateutil, botocore, s3transfer, pyasn1, rsa, PyYAML, awscli
  Running setup.py install for PyYAML ... done
Successfully installed PyYAML-3.12 awscli-1.11.185 botocore-1.7.43 colorama-0.3.7 docutils-0.14 futures-3.1.1 jmespath-0.9.3 pyasn1-0.3.7 python-dateutil-2.6.1 rsa-3.4.2 s3transfer-0.1.11 six-1.11.0

➜  ~ aws --version
zsh: command not found: aws

➜  ~ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.13.1
BuildVersion:   17B48

This does not work.

Aws on Miro's World

How klassify.com went live: one static page, GitHub Pages and Route 53

Reference Architecture as a Code

Many shapes of Architecture #

History of Faith

The beginnings #

All Day DevOps 2019

All Day DevOps 2019 - Notes #

AWS Inspection

Bunch of one-liners for AWS #

How many instances of each type do I have, and in what states? #

What CIDRs have Ingress Access to which Ports? #

Which Services am I using? #

CIDR access to ports #

Lambda runtimes #

Memory size #

Lambda Environment variables #