Wednesday, August 12, 2020

Boto3: Client vs Resource vs Session

 Here's some more detailed information on what Client, Resource, and Session are all about.

Client:

  • low-level AWS service access
  • generated from AWS service description
  • exposes botocore client to the developer
  • typically maps 1:1 with the AWS service API
  • all AWS service operations are supported by clients
  • snake-cased method names (e.g. ListBuckets API => list_buckets method)

Here's an example of client-level access to an S3 bucket's objects (at most 1000**):

import boto3

client = boto3.client('s3')
response = client.list_objects_v2(Bucket='mybucket')
for content in response['Contents']:
    obj_dict = client.get_object(Bucket='mybucket', Key=content['Key'])
    print(content['Key'], obj_dict['LastModified'])

** you would have to use a paginator, or implement your own loop, calling list_objects() repeatedly with a continuation marker if there were more than 1000.

Resource:

  • higher-level, object-oriented API
  • generated from resource description
  • uses identifiers and attributes
  • has actions (operations on resources)
  • exposes subresources and collections of AWS resources
  • does not provide 100% API coverage of AWS services

Reference: 

https://stackoverflow.com/questions/42809096/difference-in-boto3-between-resource-client-and-session


Here's the equivalent example using resource-level access to an S3 bucket's objects (all):

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('mybucket')
for obj in bucket.objects.all():
    print(obj.key, obj.last_modified)

Note that in this case you do not have to make a second API call to get the objects; they're available to you as a collection on the bucket. These collections of subresources are lazily-loaded.

You can see that the Resource version of the code is much simpler, more compact, and has more capability (it does pagination for you). The Client version of the code would actually be more complicated than shown above if you wanted to include pagination.

Session:

  • stores configuration information (primarily credentials and selected region)
  • allows you to create service clients and resources
  • boto3 creates a default session for you when needed

A useful resource to learn more about these boto3 concepts is the introductory re:Invent video.

Python argpars

 https://stackoverflow.com/questions/24180527/argparse-required-arguments-listed-under-optional-arguments


In general, the argparse module assumes that flags like -f and --bar indicate optional arguments, which can always be omitted at the command line.

Note: Required options are generally considered bad form because users expect options to be optional, and thus they should be avoided when possible.


parser = argparse.ArgumentParser(description='Foo')
parser.add_argument('-o', '--output', help='Output file name', default='stdout')
requiredNamed = parser.add_argument_group('required named arguments')
requiredNamed.add_argument('-i', '--input', help='Input file name', required=True)
parser.parse_args(['-h'])


Sunday, June 28, 2020

Vault: cannot validate certificate for 12 7.0.0.1


Problem:

$ vault statusError checking seal status: Get https://127.0.0.1:8200/v1/sys/seal-status: x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs
Cause:Command vault status try to check status with URL https://127.0.0.1:8200/v1/auth/token/lookup-selfbut it cannot validate the self-signed certificate.
Workaround:
$ export VAULT_SKIP_VERIFY=1$ vault statusKey Value--- -----Seal Type shamirInitialized trueSealed trueTotal Shares 3Threshold 3Unseal Progress 0/3Unseal Nonce n/aVersion 1.4.2HA Enabled false

Friday, April 17, 2020

Eclipse: Set project compiler compliance settings to 1.5 Set project JRE build path entry to 'J2SE-1.5'

Problem:
Set project compiler compliance settings to 1.5
Set project JRE build path entry to 'J2SE-1.5'

Cause:
Project compiler compliance is set to lower than 1.5.

Solution:

Goto Project Properties -> Java Compiler,  you will see JDK Compliance in the right panel,  Right beneath it:


uncheck "Use compliance from execution environment 'CDC-1.0/Foundation-1.0' on the 'Java Build Path'", then choose proper Compiler compliance level:


Tuesday, July 2, 2019

Clockwise Rotating Text

<!DOCTYPE html><html lang='en' class=''>
<head>
<style class="cp-pen-styles">#container {
  margin: 0%;
}

#circle {
  position: relative;
  width: 100%;
  padding-bottom: 100%;
  overflow: hidden;
}

#circle text {
  font-family: "Helvetica Neue", Arial;
  font-size: 16px;
  font-weight: bold;
}

#circle svg {
  position: absolute;
  left: 0;
  top: 0;
  width: 100%;
  height: 200px;

  -webkit-animation-name: rotate;
  -moz-animation-name: rotate;
  -ms-animation-name: rotate;
  -o-animation-name: rotate;
  animation-name: rotate;
  -webkit-animation-duration: 5s;
  -moz-animation-duration: 5s;
  -ms-animation-duration: 5s;
  -o-animation-duration: 5s;
  animation-duration: 5s;
  -webkit-animation-iteration-count: infinite;
  -moz-animation-iteration-count: infinite;
  -ms-animation-iteration-count: infinite;
  -o-animation-iteration-count: infinite;
  animation-iteration-count: infinite;
  -webkit-animation-timing-function: linear;
  -moz-animation-timing-function: linear;
  -ms-animation-timing-function: linear;
  -o-animation-timing-function: linear;
  animation-timing-function: linear;
}

@keyframes rotate {
  from {
    transform: rotate(0);
  }
  to {
    transform: rotate(360deg);
  }
}
</style>
<style>
body {
  #background-color: #333;
  font-family: 'Luckiest Guy', cursive;
  font-size: 35px;
}

path {
  fill: #CCEEFF;
}

text {
  fill: #1d8703;
}
</style>

</head><body>
<div id="container">
<div id="circle">
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" width="200px" height="200px" viewBox="0 0 200 200" enable-background="new 0 0 200 200" xml:space="preserve">
<defs>
<path id="circlePath"
d="
M 100, 100
m 60, 0
a -60,60 0 0,0 -120,0
a -60,60 0 0,0 120,0
"
/>
</defs>
<g>
<use xlink:href="#circlePath" fill="#CCEEFF" />
<text class="text">
<textPath xlink:href="#circlePath"> <<<<<<< A R T H U R R U N N I N G <<<<<<< </textPath>
</text>
</g>
</svg>
</div>
</div>
<desc><path id="circlePath" d="M 100, 100 m -60, 0 a 60,60 0 0,0 120,0 a 60,60 0 0,0 -120,0 " /></desc>
</body></html>

Wednesday, May 1, 2019

Ansible - Create User with Password

To Create User with Password, the password must be encrypted.
 
Reference: https://docs.python.org/3/library/crypt.html 
 

   # created with:
   # python -c 'import crypt; print crypt.crypt("sesame", "$1$opendoor$")'
   - name: check if group exists
     shell: /usr/bin/getent group $group | /usr/bin/wc -l | tr -d ' '
     register: group_exist

   - name: check if user exists
     shell: /usr/bin/getent passwd $user | /usr/bin/wc -l | tr -d ' '
     register: user_exist

 
   - name: Add user "aniu" to the remote server
     remote_user: root
     user:
       name: apple
       password: $1$opendoor$oFO4rBXnUmwP3W3rK859N.
       comment: "Orange Sales"
       uid: 5571
       group: users
       append: yes
       shell: /bin/bash
       generate_ssh_key: yes
       ssh_key_bits: 2048
       ssh_key_file: .ssh/id_rsa
     only_if: ${user_exist.stdout} == 0