RTSP Server gst-rtsp-server on Amazon Linux 2

環境

# cat /etc/system-release
Amazon Linux release 2 (Karoo)
# uname -r
4.14.203-156.332.amzn2.x86_64

gst-rtsp-serverのインストール

$ git clone git://anongit.freedesktop.org/gstreamer/gst-rtsp-server
$ cd gst-rtsp-server/
$ git checkout 1.4
$ ./autogen.sh
$ make
$ sudo make install

autogen.sh 実行に必要だったパッケージ

$ sudo yum install git
$ sudo yum install libtool
$ sudo yum install gettext-devel
$ sudo yum install gtk-doc
$ sudo yum install gcc-c++
$ sudo yum install python3
$ sudo python3 -m pip install meson
$ sudo python3 -m pip install ninja
$ sudo yum install gstreamer1
$ sudo yum install gstreamer1-devel
$ sudo yum install gstreamer1-plugins-base
$ sudo yum install gstreamer1-plugins-base-devel
# 後に下記インストール
$ sudo yum install gstreamer1-devel gstreamer1-plugins-base-tools gstreamer1-devel-docs gstreamer1-plugins-base-devel gstreamer1-plugins-base-devel-docs gstreamer1-plugins-good gstreamer1-plugins-good-extras gstreamer1-plugins-ugly gstreamer1-plugins-ugly-devel-docs  gstreamer1-plugins-bad-free gstreamer1-plugins-bad-free-devel gstreamer1-plugins-bad-free-extras

# cd /usr/local/src/
# wget http://ftp.gnome.org/pub/gnome/sources/glib/2.64/glib-2.64.4.tar.xz
# tar Jxvf glib-2.64.4.tar.xz
# cd glib-2.64.4/
# meson _build
# ninja -C _build
# less INSTALL.in
# ninja -C _build install
# pkg-config glib-2.0 --modversion

vi ~/.bash_profile
export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig

x264encインストール

# cd /usr/local/src/
# wget https://code.videolan.org/videolan/x264/-/archive/master/x264-master.tar.bz2
# tar jxvf x264-master.tar.bz2
# cd ../x264-master/
# ./configure --enable-shared
# make
# make install
# gst-inspect-1.0 x264enc

必要だったパッケージ

# cd /usr/local/src/
# wget https://www.nasm.us/pub/nasm/releasebuilds/2.15.05/nasm-2.15.05.tar.xz
# tar Jxvf nasm-2.15.05.tar.xz
# cd nasm-2.15.05/
# ./configure
# make
# make install

$ sudo yum install gstreamer1.0-plugins-ugly
$ sudo yum install gstreamer1-plugins-ugly
$ sudo yum install gstreamer1-plugins-ugly-devel-docs
# 後に下記インストール
$ sudo yum install gstreamer1-devel gstreamer1-plugins-base-tools gstreamer1-devel-docs gstreamer1-plugins-base-devel gstreamer1-plugins-base-devel-docs gstreamer1-plugins-good gstreamer1-plugins-good-extras gstreamer1-plugins-ugly gstreamer1-plugins-ugly-devel-docs  gstreamer1-plugins-bad-free gstreamer1-plugins-bad-free-devel gstreamer1-plugins-bad-free-extras

ffmpeg（直接不要だったかも）

# wget http://mirror.centos.org/centos/7/os/x86_64/Packages/libva-1.8.3-1.el7.x86_64.rpm
# rpm -ivh libva-1.8.3-1.el7.x86_64.rpm
# wget https://download-ib01.fedoraproject.org/pub/epel/7/x86_64/Packages/l/libass-0.13.4-6.el7.x86_64.rpm
# rpm -ivh libass-0.13.4-6.el7.x86_64.rpm

$ sudo amazon-linux-extras install epel
$ sudo rpm --import http://li.nux.ro/download/nux/RPM-GPG-KEY-nux.ro
$ sudo rpm -Uvh http://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-1.el7.nux.noarch.rpm
$ sudo yum install ffmpeg ffmpeg-devel

テスト動画の再生

$ ./test-launch '( videotestsrc ! x264enc ! rtph264pay name=pay0 pt=96 )'

AWS Batch コンピューティング環境の終了タイミング

コメントを残す

実動作では何もジョブが実行されない場合に、10分で terminated されていた。

コンピューティング環境一つに対して、Auto Scalingグループが作成されており、

次のログが出ていた。

Terminating EC2 instance: i-0a0d23067827fb809

2019 July 5 03:17:14 UTC+9

2019 July 5 03:18:19 UTC+9

説明:説明Terminating EC2 instance: i-0a0d23067827fb809

原因:原因At 2019-07-04T18:17:14Z instance i-0a0d23067827fb809 was taken out of service in response to a user request, shrinking the capacity from 1 to 0.

下記だろうか。外部が指定している？AWS Batchが指定している？

https://docs.aws.amazon.com/ja_jp/autoscaling/ec2/APIReference/API_TerminateInstanceInAutoScalingGroup.html

クールダウンの300秒が関係している？

他、

AWS Batchを使って5分以上かかる処理を実行してみる

調査中。

AWS Batch　起動AMIのユーザデータ

コメントを残す

確認したAMI

amzn-ami-2018.03.u-amazon-ecs-optimized (ami-0e1aa8c2e9d719f58)

起動AMIから確認。

<br />Content-Type: multipart/mixed; boundary="==BOUNDARY=="
MIME-Version: 1.0

--==BOUNDARY==
MIME-Version: 1.0
Content-Type: text/x-shellscript; charset="us-ascii"

#!/bin/bash
echo ECS_CLUSTER=first-run-compute-environment-minimum_Batch_049f0a53-0a37-37c7-8f7d-9910330bdbec&gt;&gt;/etc/ecs/ecs.config
echo ECS_DISABLE_IMAGE_CLEANUP=false&gt;&gt;/etc/ecs/ecs.config
echo ECS_ENGINE_TASK_CLEANUP_WAIT_DURATION=2m&gt;&gt;/etc/ecs/ecs.config
echo ECS_IMAGE_CLEANUP_INTERVAL=10m&gt;&gt;/etc/ecs/ecs.config
echo ECS_IMAGE_MINIMUM_CLEANUP_AGE=10m&gt;&gt;/etc/ecs/ecs.config
echo ECS_NUM_IMAGES_DELETE_PER_CYCLE=5&gt;&gt;/etc/ecs/ecs.config
echo ECS_RESERVED_MEMORY=32&gt;&gt;/etc/ecs/ecs.config

--==BOUNDARY==--

<br />Content-Type: multipart/mixed; boundary="==BOUNDARY=="

MIME-Version: 1.0

--==BOUNDARY==

MIME-Version: 1.0

Content-Type: text/x-shellscript; charset="us-ascii"

#!/bin/bash

echo ECS_CLUSTER=first-run-compute-environment-minimum_Batch_049f0a53-0a37-37c7-8f7d-9910330bdbec>>/etc/ecs/ecs.config

echo ECS_DISABLE_IMAGE_CLEANUP=false>>/etc/ecs/ecs.config

echo ECS_ENGINE_TASK_CLEANUP_WAIT_DURATION=2m>>/etc/ecs/ecs.config

echo ECS_IMAGE_CLEANUP_INTERVAL=10m>>/etc/ecs/ecs.config

echo ECS_IMAGE_MINIMUM_CLEANUP_AGE=10m>>/etc/ecs/ecs.config

echo ECS_NUM_IMAGES_DELETE_PER_CYCLE=5>>/etc/ecs/ecs.config

echo ECS_RESERVED_MEMORY=32>>/etc/ecs/ecs.config

--==BOUNDARY==--

Amazon ECR

コメントを残す

https://docs.aws.amazon.com/ja_jp/AmazonECR/latest/userguide/docker-basics.html
に従って実施

1008 touch Dockerfile
1009 vi Dockerfile

1010 docker build -t hello-world .
1011 docker images --filter reference=hello-world

1014 docker run -t -i -p 10080:80 hello-world

1017 aws ecr create-repository --repository-name hello-repository --region ap-northeast-1
1018 docker tag hello-world xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/hello-repository

1020 aws --version
1021 docker --version
1025 pip install --upgrade pip --user
1026 pip install awscli --upgrade --user
# version up

1028 aws ecr get-login --no-include-email --region ap-northeast-1
1029 docker login -u AWS -p xxxxxxxxxx https://xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com
1030 docker push xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/hello-repository

1008 touch Dockerfile

1009 vi Dockerfile

1010 docker build -t hello-world .

1011 docker images --filter reference=hello-world

1014 docker run -t -i -p 10080:80 hello-world

1017 aws ecr create-repository --repository-name hello-repository --region ap-northeast-1

1018 docker tag hello-world xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/hello-repository

1020 aws --version

1021 docker --version

1025 pip install --upgrade pip --user

1026 pip install awscli --upgrade --user

# version up

1028 aws ecr get-login --no-include-email --region ap-northeast-1

1029 docker login -u AWS -p xxxxxxxxxx https://xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com

1030 docker push xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/hello-repository

Lambda Python runtime environment

コメントを残す

python3.7

Response:
0

Request ID:
"50b03c17-4f23-4dee-b449-2d359f7b4e14"

Function Logs:
START RequestId: 50b03c17-4f23-4dee-b449-2d359f7b4e14 Version: $LATEST
===[environment-variables]===
PATH
LD_LIBRARY_PATH
LANG
TZ
LAMBDA_TASK_ROOT
LAMBDA_RUNTIME_DIR
AWS_REGION
AWS_DEFAULT_REGION
AWS_LAMBDA_LOG_GROUP_NAME
AWS_LAMBDA_LOG_STREAM_NAME
AWS_LAMBDA_FUNCTION_NAME
AWS_LAMBDA_FUNCTION_MEMORY_SIZE
AWS_LAMBDA_FUNCTION_VERSION
_AWS_XRAY_DAEMON_ADDRESS
_AWS_XRAY_DAEMON_PORT
AWS_XRAY_DAEMON_ADDRESS
AWS_XRAY_CONTEXT_MISSING
AWS_EXECUTION_ENV
_HANDLER
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN
_X_AMZN_TRACE_ID

===[runtime directory:/var/runtime]===
python_dateutil-2.7.5.dist-info
docutils-0.14.dist-info
lambda_runtime_marshaller.py
s3transfer-0.1.13.dist-info
test_bootstrap.py
runtime-release
bootstrap.py
six-1.11.0.dist-info
urllib3-1.24.1.dist-info
s3transfer
lambda_runtime_exception.py
botocore
dateutil
bin
docutils
six.py
botocore-1.12.42.dist-info
urllib3
lambda_runtime_client.py
boto3-1.9.42.dist-info
boto3
bootstrap
test_lambda_runtime_client.py
jmespath
__pycache__
test_lambda_runtime_marshaller.py
jmespath-0.9.3.dist-info
===[amazon linux version]===
Amazon Linux AMI release 2017.03
0
END RequestId: 50b03c17-4f23-4dee-b449-2d359f7b4e14
REPORT RequestId: 50b03c17-4f23-4dee-b449-2d359f7b4e14  Duration: 109.71 ms Billed Duration: 200 ms     Memory Size: 128 MB Max Memory Used: 55 MB

Response:

Request ID:

"50b03c17-4f23-4dee-b449-2d359f7b4e14"

Function Logs:

START RequestId: 50b03c17-4f23-4dee-b449-2d359f7b4e14 Version: $LATEST

===[environment-variables]===

PATH

LD_LIBRARY_PATH

LANG

LAMBDA_TASK_ROOT

LAMBDA_RUNTIME_DIR

AWS_REGION

AWS_DEFAULT_REGION

AWS_LAMBDA_LOG_GROUP_NAME

AWS_LAMBDA_LOG_STREAM_NAME

AWS_LAMBDA_FUNCTION_NAME

AWS_LAMBDA_FUNCTION_MEMORY_SIZE

AWS_LAMBDA_FUNCTION_VERSION

_AWS_XRAY_DAEMON_ADDRESS

_AWS_XRAY_DAEMON_PORT

AWS_XRAY_DAEMON_ADDRESS

AWS_XRAY_CONTEXT_MISSING

AWS_EXECUTION_ENV

_HANDLER

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

AWS_SESSION_TOKEN

_X_AMZN_TRACE_ID

===[runtime directory:/var/runtime]===

python_dateutil-2.7.5.dist-info

docutils-0.14.dist-info

lambda_runtime_marshaller.py

s3transfer-0.1.13.dist-info

test_bootstrap.py

runtime-release

bootstrap.py

six-1.11.0.dist-info

urllib3-1.24.1.dist-info

s3transfer

lambda_runtime_exception.py

botocore

dateutil

bin

docutils

six.py

botocore-1.12.42.dist-info

urllib3

lambda_runtime_client.py

boto3-1.9.42.dist-info

boto3

bootstrap

test_lambda_runtime_client.py

jmespath

__pycache__

test_lambda_runtime_marshaller.py

jmespath-0.9.3.dist-info

===[amazon linux version]===

Amazon Linux AMI release 2017.03

END RequestId: 50b03c17-4f23-4dee-b449-2d359f7b4e14

REPORT RequestId: 50b03c17-4f23-4dee-b449-2d359f7b4e14 Duration: 109.71 ms Billed Duration: 200 ms Memory Size: 128 MB Max Memory Used: 55 MB

python3.6

Response:
{
  "errorMessage": "[Errno 2] No such file or directory: 'cat /etc/system-release': 'cat /etc/system-release'",
  "errorType": "FileNotFoundError",
  "stackTrace": [
    [
      "/var/task/lambda_function.py",
      22,
      "lambda_handler",
      "ret = subprocess.call(cmd)"
    ],
    [
      "/var/lang/lib/python3.6/subprocess.py",
      287,
      "call",
      "with Popen(*popenargs, **kwargs) as p:"
    ],
    [
      "/var/lang/lib/python3.6/subprocess.py",
      729,
      "__init__",
      "restore_signals, start_new_session)"
    ],
    [
      "/var/lang/lib/python3.6/subprocess.py",
      1364,
      "_execute_child",
      "raise child_exception_type(errno_num, err_msg, err_filename)"
    ]
  ]
}

Request ID:
"310363ce-1c33-4c07-9d31-9b445790422e"

Function Logs:
START RequestId: 310363ce-1c33-4c07-9d31-9b445790422e Version: $LATEST
===[environment-variables]===
PATH
LD_LIBRARY_PATH
LANG
TZ
LAMBDA_TASK_ROOT
LAMBDA_RUNTIME_DIR
AWS_REGION
AWS_DEFAULT_REGION
AWS_LAMBDA_LOG_GROUP_NAME
AWS_LAMBDA_LOG_STREAM_NAME
AWS_LAMBDA_FUNCTION_NAME
AWS_LAMBDA_FUNCTION_MEMORY_SIZE
AWS_LAMBDA_FUNCTION_VERSION
_AWS_XRAY_DAEMON_ADDRESS
_AWS_XRAY_DAEMON_PORT
AWS_XRAY_DAEMON_ADDRESS
AWS_XRAY_CONTEXT_MISSING
_X_AMZN_TRACE_ID
AWS_EXECUTION_ENV
_HANDLER
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN
AWS_SECURITY_TOKEN
PYTHONPATH

===[runtime directory:/var/runtime]===
liblambdaruntime.so
docutils-0.14.dist-info
botocore-1.10.74.dist-info
s3transfer-0.1.13.dist-info
six-1.11.0.dist-info
boto3-1.7.74.dist-info
s3transfer
botocore
liblambdalog.so
dateutil
awslambda
docutils
six.py
boto3
liblambdaio.so
jmespath
__pycache__
python_dateutil-2.7.3.dist-info
jmespath-0.9.3.dist-info
liblambdaipc.so

===[amazon linux version]===
[Errno 2] No such file or directory: 'cat /etc/system-release': 'cat /etc/system-release': FileNotFoundError
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 22, in lambda_handler
    ret = subprocess.call(cmd)
  File "/var/lang/lib/python3.6/subprocess.py", line 287, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/var/lang/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/var/lang/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'cat /etc/system-release': 'cat /etc/system-release'

END RequestId: 310363ce-1c33-4c07-9d31-9b445790422e
REPORT RequestId: 310363ce-1c33-4c07-9d31-9b445790422e  Duration: 50.25 ms  Billed Duration: 100 ms     Memory Size: 128 MB Max Memory Used: 49 MB

100

101

Response:

{

"errorMessage": "[Errno 2] No such file or directory: 'cat /etc/system-release': 'cat /etc/system-release'",

"errorType": "FileNotFoundError",

"stackTrace": [

[

"/var/task/lambda_function.py",

22,

"lambda_handler",

"ret = subprocess.call(cmd)"

[

"/var/lang/lib/python3.6/subprocess.py",

287,

"call",

"with Popen(*popenargs, **kwargs) as p:"

[

"/var/lang/lib/python3.6/subprocess.py",

729,

"__init__",

"restore_signals, start_new_session)"

[

"/var/lang/lib/python3.6/subprocess.py",

1364,

"_execute_child",

"raise child_exception_type(errno_num, err_msg, err_filename)"

]

}

Request ID:

"310363ce-1c33-4c07-9d31-9b445790422e"

Function Logs:

START RequestId: 310363ce-1c33-4c07-9d31-9b445790422e Version: $LATEST

===[environment-variables]===

PATH

LD_LIBRARY_PATH

LANG

LAMBDA_TASK_ROOT

LAMBDA_RUNTIME_DIR

AWS_REGION

AWS_DEFAULT_REGION

AWS_LAMBDA_LOG_GROUP_NAME

AWS_LAMBDA_LOG_STREAM_NAME

AWS_LAMBDA_FUNCTION_NAME

AWS_LAMBDA_FUNCTION_MEMORY_SIZE

AWS_LAMBDA_FUNCTION_VERSION

_AWS_XRAY_DAEMON_ADDRESS

_AWS_XRAY_DAEMON_PORT

AWS_XRAY_DAEMON_ADDRESS

AWS_XRAY_CONTEXT_MISSING

_X_AMZN_TRACE_ID

AWS_EXECUTION_ENV

_HANDLER

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

AWS_SESSION_TOKEN

AWS_SECURITY_TOKEN

PYTHONPATH

===[runtime directory:/var/runtime]===

liblambdaruntime.so

docutils-0.14.dist-info

botocore-1.10.74.dist-info

s3transfer-0.1.13.dist-info

six-1.11.0.dist-info

boto3-1.7.74.dist-info

s3transfer

botocore

liblambdalog.so

dateutil

awslambda

docutils

six.py

boto3

liblambdaio.so

jmespath

__pycache__

python_dateutil-2.7.3.dist-info

jmespath-0.9.3.dist-info

liblambdaipc.so

===[amazon linux version]===

[Errno 2] No such file or directory: 'cat /etc/system-release': 'cat /etc/system-release': FileNotFoundError

Traceback (most recent call last):

File "/var/task/lambda_function.py", line 22, in lambda_handler

ret = subprocess.call(cmd)

File "/var/lang/lib/python3.6/subprocess.py", line 287, in call

with Popen(*popenargs, **kwargs) as p:

File "/var/lang/lib/python3.6/subprocess.py", line 729, in __init__

restore_signals, start_new_session)

File "/var/lang/lib/python3.6/subprocess.py", line 1364, in _execute_child

raise child_exception_type(errno_num, err_msg, err_filename)

FileNotFoundError: [Errno 2] No such file or directory: 'cat /etc/system-release': 'cat /etc/system-release'

END RequestId: 310363ce-1c33-4c07-9d31-9b445790422e

REPORT RequestId: 310363ce-1c33-4c07-9d31-9b445790422e Duration: 50.25 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 49 MB

Python2.7

Response:
{
  "stackTrace": [
    [
      "/var/task/lambda_function.py",
      24,
      "lambda_handler",
      "ret = subprocess.call(cmd)"
    ],
    [
      "/usr/lib64/python2.7/subprocess.py",
      523,
      "call",
      "return Popen(*popenargs, **kwargs).wait()"
    ],
    [
      "/usr/lib64/python2.7/subprocess.py",
      711,
      "__init__",
      "errread, errwrite)"
    ],
    [
      "/usr/lib64/python2.7/subprocess.py",
      1343,
      "_execute_child",
      "raise child_exception"
    ]
  ],
  "errorType": "OSError",
  "errorMessage": "[Errno 2] No such file or directory"
}

Request ID:
"098c848f-87e7-45dd-89fd-4b79d0767907"

Function Logs:
START RequestId: 098c848f-87e7-45dd-89fd-4b79d0767907 Version: $LATEST
===[environment-variables]===
_HANDLER
AWS_LAMBDA_FUNCTION_VERSION
LAMBDA_TASK_ROOT
PATH
LD_LIBRARY_PATH
LANG
TZ
AWS_REGION
AWS_XRAY_CONTEXT_MISSING
AWS_SESSION_TOKEN
AWS_SECURITY_TOKEN
LAMBDA_RUNTIME_DIR
PYTHONPATH
AWS_LAMBDA_FUNCTION_MEMORY_SIZE
_AWS_XRAY_DAEMON_PORT
_AWS_XRAY_DAEMON_ADDRESS
AWS_LAMBDA_LOG_GROUP_NAME
AWS_LAMBDA_LOG_STREAM_NAME
AWS_ACCESS_KEY_ID
_X_AMZN_TRACE_ID
AWS_DEFAULT_REGION
AWS_SECRET_ACCESS_KEY
AWS_EXECUTION_ENV
AWS_XRAY_DAEMON_ADDRESS
AWS_LAMBDA_FUNCTION_NAME

===[runtime directory:/var/runtime]===
liblambdaruntime.so
docutils-0.14.dist-info
botocore-1.10.74.dist-info
futures-3.2.0.dist-info
s3transfer-0.1.13.dist-info
six-1.11.0.dist-info
boto3-1.7.74.dist-info
s3transfer
botocore
liblambdalog.so
dateutil
six.pyc
awslambda
bin
docutils
six.py
boto3
liblambdaio.so
jmespath
concurrent
python_dateutil-2.7.3.dist-info
jmespath-0.9.3.dist-info
liblambdaipc.so

===[amazon linux version]===
[Errno 2] No such file or directory: OSError
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 24, in lambda_handler
    ret = subprocess.call(cmd)
  File "/usr/lib64/python2.7/subprocess.py", line 523, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

END RequestId: 098c848f-87e7-45dd-89fd-4b79d0767907
REPORT RequestId: 098c848f-87e7-45dd-89fd-4b79d0767907  Duration: 93.64 ms  Billed Duration: 100 ms     Memory Size: 128 MB Max Memory Used: 42 MB

100

101

102

103

104

Response:

{

"stackTrace": [

[

"/var/task/lambda_function.py",

24,

"lambda_handler",

"ret = subprocess.call(cmd)"

[

"/usr/lib64/python2.7/subprocess.py",

523,

"call",

"return Popen(*popenargs, **kwargs).wait()"

[

"/usr/lib64/python2.7/subprocess.py",

711,

"__init__",

"errread, errwrite)"

[

"/usr/lib64/python2.7/subprocess.py",

1343,

"_execute_child",

"raise child_exception"

]

"errorType": "OSError",

"errorMessage": "[Errno 2] No such file or directory"

}

Request ID:

"098c848f-87e7-45dd-89fd-4b79d0767907"

Function Logs:

START RequestId: 098c848f-87e7-45dd-89fd-4b79d0767907 Version: $LATEST

===[environment-variables]===

_HANDLER

AWS_LAMBDA_FUNCTION_VERSION

LAMBDA_TASK_ROOT

PATH

LD_LIBRARY_PATH

LANG

AWS_REGION

AWS_XRAY_CONTEXT_MISSING

AWS_SESSION_TOKEN

AWS_SECURITY_TOKEN

LAMBDA_RUNTIME_DIR

PYTHONPATH

AWS_LAMBDA_FUNCTION_MEMORY_SIZE

_AWS_XRAY_DAEMON_PORT

_AWS_XRAY_DAEMON_ADDRESS

AWS_LAMBDA_LOG_GROUP_NAME

AWS_LAMBDA_LOG_STREAM_NAME

AWS_ACCESS_KEY_ID

_X_AMZN_TRACE_ID

AWS_DEFAULT_REGION

AWS_SECRET_ACCESS_KEY

AWS_EXECUTION_ENV

AWS_XRAY_DAEMON_ADDRESS

AWS_LAMBDA_FUNCTION_NAME

===[runtime directory:/var/runtime]===

liblambdaruntime.so

docutils-0.14.dist-info

botocore-1.10.74.dist-info

futures-3.2.0.dist-info

s3transfer-0.1.13.dist-info

six-1.11.0.dist-info

boto3-1.7.74.dist-info

s3transfer

botocore

liblambdalog.so

dateutil

six.pyc

awslambda

bin

docutils

six.py

boto3

liblambdaio.so

jmespath

concurrent

python_dateutil-2.7.3.dist-info

jmespath-0.9.3.dist-info

liblambdaipc.so

===[amazon linux version]===

[Errno 2] No such file or directory: OSError

Traceback (most recent call last):

File "/var/task/lambda_function.py", line 24, in lambda_handler

ret = subprocess.call(cmd)

File "/usr/lib64/python2.7/subprocess.py", line 523, in call

return Popen(*popenargs, **kwargs).wait()

File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__

errread, errwrite)

File "/usr/lib64/python2.7/subprocess.py", line 1343, in _execute_child

raise child_exception

OSError: [Errno 2] No such file or directory

END RequestId: 098c848f-87e7-45dd-89fd-4b79d0767907

REPORT RequestId: 098c848f-87e7-45dd-89fd-4b79d0767907 Duration: 93.64 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 42 MB

2.7でもboto3-1.7.74.dist-info

Lambda pythonバージョン

https://docs.aws.amazon.com/ja_jp/lambda/latest/dg/python-programming-model.html

boto3 s3 署名バージョン対応

1.5.71 (Botocore)、1.4.6 (Boto3) にアップグレード。

【超重要】対応しないと使えなくなるかも？！今、全S3ユーザがチェックすべき署名バージョン２の廃止について

pythonコード

# -*- coding: utf-8 -*-
import os
import subprocess

def lambda_handler(event, context):

    # 環境変数一覧を取得
    print("===[environment-variables]===")
    for env in os.environ:
        print(env)

    #ランタイムの格納されているディレクトリを確認
    rtdir = os.environ.get("LAMBDA_RUNTIME_DIR")
    print("\n===[runtime directory:"+rtdir+"]===")
    
    #ディレクトリ内を表示
    files = os.listdir(rtdir)
    for file in files:
        print(file)

    # amazon linux version
    print("\n===[amazon linux version]===")
    cmd = "cat /etc/system-release";
    ret = subprocess.call(cmd)
    print(ret)

    cmd = "ls -la /etc/system*";
    ret = subprocess.call(cmd)
    print(ret)

    return 0

# -*- coding: utf-8 -*-

import os

import subprocess

def lambda_handler(event, context):

# 環境変数一覧を取得

print("===[environment-variables]===")

for env in os.environ:

print(env)

#ランタイムの格納されているディレクトリを確認

rtdir = os.environ.get("LAMBDA_RUNTIME_DIR")

print("\n===[runtime directory:"+rtdir+"]===")

#ディレクトリ内を表示

files = os.listdir(rtdir)

for file in files:

print(file)

# amazon linux version

print("\n===[amazon linux version]===")

cmd = "cat /etc/system-release";

ret = subprocess.call(cmd)

print(ret)

cmd = "ls -la /etc/system*";

ret = subprocess.call(cmd)

print(ret)

return 0

https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/UsingAWSSDK.html

すべての AWS リージョンでは、AWS SDK はデフォルトで署名バージョン 4 を使用してリクエストを認証します。2016 年 5 月以前にリリースされた AWS SDK を使用する場合、次の表に示すように、署名バージョン 4 のリクエストが必要になることがあります。

[s3] use-sigv4 = True

5af85bf0

AWS SDK for PHPでのslab増大とNSS_SDB_USE_CACHE対策の効果確認

コメントを残す

色々調べると、GuzzleHttpがHTTPSで通信する際に増えるらしい

まとめ

AWS SDKのクライアント、GuzzleHttpも最終的にphpのcurl_～関数を呼んでいる
※constructorでcurlが無い場合、Exception吐くコードがあるし、AWSの要件にもcurlが入っている
php curl の nssは多分、ビルドされたcurlのNSSバージョンに依存してそう
※curl -V のNSS/3.27 の部分
curlをmakeすると、その時のOSのnssバージョンが組み込まれた
curl 7.19.7 (x86_64-unknown-linux-gnu) libcurl/7.19.7 NSS/3.36
かつ、その際、nss-softokn-3.14.3-23.3.el6_8.x86_64 で、curlコマンドのNSS_SDB_USE_CACHE=yes は有効だった
phpのcurlバージョンは、コマンドと同じくlibcurlに依存してそうだが、curl_version(); 又は、phpinfo(); で確認できる

各種情報

[root@ip-172-31-16-82 centos]# cat /etc/redhat-release
CentOS release 6.9 (Final)
[root@ip-172-31-16-82 centos]# rpm -qa | grep nss-softokn
nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64
nss-softokn-3.14.3-23.3.el6_8.x86_64
[root@ip-172-31-16-82 centos]# php -v
PHP 5.5.38 (cli) (built: Mar  1 2018 11:12:35)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies


[centos@ip-172-31-16-82 slab-nss]$ cat composer.json
{
    "require": {
        "aws/aws-sdk-php": "^2.8"
    }
}

[root@ip-172-31-16-82 centos]# cat /etc/redhat-release

CentOS release 6.9 (Final)

[root@ip-172-31-16-82 centos]# rpm -qa | grep nss-softokn

nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64

nss-softokn-3.14.3-23.3.el6_8.x86_64

[root@ip-172-31-16-82 centos]# php -v

PHP 5.5.38 (cli) (built: Mar 1 2018 11:12:35)

[centos@ip-172-31-16-82 slab-nss]$ cat composer.json

{

"require": {

"aws/aws-sdk-php": "^2.8"

}

AWS-SDKでSNSへpublishするテストコードを準備

<?php

require_once('./vendor/autoload.php');
use Aws\Sns\SnsClient;

$arn = "arn:aws:sns:ap-northeast-1:xxxxxx:aws-sdk-nss-dentry-test";
$message = 'dummy message';

$client = SnsClient::factory(array(
    'profile' => 'default',
    'region'  => 'ap-northeast-1',
));

for ($i = 0; $i < 100; $i ++) {
    $result = $client->publish(array(
        'TopicArn' => $arn,
        'Message' => $message
    ));
}

<?php

require_once('./vendor/autoload.php');

use Aws\Sns\SnsClient;

$arn = "arn:aws:sns:ap-northeast-1:xxxxxx:aws-sdk-nss-dentry-test";

$message = 'dummy message';

$client = SnsClient::factory(array(

'profile' => 'default',

'region' => 'ap-northeast-1',

));

for ($i = 0; $i < 100; $i ++) {

$result = $client->publish(array(

'TopicArn' => $arn,

'Message' => $message

));

}

実行前後の情報

[centos@ip-172-31-16-82 ~]$ sar -v 0
Linux 2.6.32-696.30.1.el6.x86_64 (ip-172-31-16-82.ap-northeast-1.compute.internal)      07/08/2018      _x86_64_        (1 CPU)

07:51:10 AM dentunusd   file-nr  inode-nr    pty-nr
07:51:10 AM     41344       736      7227         5
[centos@ip-172-31-16-82 ~]$ sar -v 0
Linux 2.6.32-696.30.1.el6.x86_64 (ip-172-31-16-82.ap-northeast-1.compute.internal)      07/08/2018      _x86_64_        (1 CPU)

07:51:33 AM dentunusd   file-nr  inode-nr    pty-nr
07:51:33 AM     61344       736      7227         5

[centos@ip-172-31-16-82 ~]$ slabtop -o | grep dentry
 48140  48105  99%    0.19K   2407       20      9628K dentry
[centos@ip-172-31-16-82 ~]$ slabtop -o | grep dentry
 68100  68100 100%    0.19K   3405       20     13620K dentry

[centos@ip-172-31-16-82 ~]$ free
             total       used       free     shared    buffers     cached
Mem:        499996     174200     325796        160      14212      53744
-/+ buffers/cache:     106244     393752
Swap:      1048572       1280    1047292
[centos@ip-172-31-16-82 ~]$ free
             total       used       free     shared    buffers     cached
Mem:        499996     178176     321820        160      14220      53744
-/+ buffers/cache:     110212     389784
Swap:      1048572       1280    1047292

[centos@ip-172-31-16-82 ~]$ sar -v 0

Linux 2.6.32-696.30.1.el6.x86_64 (ip-172-31-16-82.ap-northeast-1.compute.internal) 07/08/2018 _x86_64_ (1 CPU)

07:51:10 AM dentunusd file-nr inode-nr pty-nr

07:51:10 AM 41344 736 7227 5

[centos@ip-172-31-16-82 ~]$ sar -v 0

Linux 2.6.32-696.30.1.el6.x86_64 (ip-172-31-16-82.ap-northeast-1.compute.internal) 07/08/2018 _x86_64_ (1 CPU)

07:51:33 AM dentunusd file-nr inode-nr pty-nr

07:51:33 AM 61344 736 7227 5

[centos@ip-172-31-16-82 ~]$ slabtop -o | grep dentry

48140 48105 99% 0.19K 2407 20 9628K dentry

[centos@ip-172-31-16-82 ~]$ slabtop -o | grep dentry

68100 68100 100% 0.19K 3405 20 13620K dentry

[centos@ip-172-31-16-82 ~]$ free

total used free shared buffers cached

Mem: 499996 174200 325796 160 14212 53744

-/+ buffers/cache: 106244 393752

Swap: 1048572 1280 1047292

[centos@ip-172-31-16-82 ~]$ free

total used free shared buffers cached

Mem: 499996 178176 321820 160 14220 53744

-/+ buffers/cache: 110212 389784

Swap: 1048572 1280 1047292

dentunusedが増加し、dentryのslabが増加している。

buffers/cacheが減少している。

対策コードを追加

<?php
putenv("NSS_SDB_USE_CACHE=yes"); // ★追加

require_once('./vendor/autoload.php');
use Aws\Sns\SnsClient;

$arn = "arn:aws:sns:ap-northeast-1:xxxxxx:aws-sdk-nss-dentry-test";
$message = 'dummy message';

$client = SnsClient::factory(array(
    'profile' => 'default',
    'region'  => 'ap-northeast-1',
));

for ($i = 0; $i < 100; $i ++) {
    $result = $client->publish(array(
        'TopicArn' => $arn,
        'Message' => $message
    ));
}

<?php

putenv("NSS_SDB_USE_CACHE=yes"); // ★追加

require_once('./vendor/autoload.php');

use Aws\Sns\SnsClient;

$arn = "arn:aws:sns:ap-northeast-1:xxxxxx:aws-sdk-nss-dentry-test";

$message = 'dummy message';

$client = SnsClient::factory(array(

'profile' => 'default',

'region' => 'ap-northeast-1',

));

for ($i = 0; $i < 100; $i ++) {

$result = $client->publish(array(

'TopicArn' => $arn,

'Message' => $message

));

}

実行前後の情報

[centos@ip-172-31-16-82 ~]$ sar -v 0
Linux 2.6.32-696.30.1.el6.x86_64 (ip-172-31-16-82.ap-northeast-1.compute.internal)      07/08/2018      _x86_64_        (1 CPU)

07:55:05 AM dentunusd   file-nr  inode-nr    pty-nr
07:55:05 AM     61349       736      7230         5
[centos@ip-172-31-16-82 ~]$ sar -v 0
Linux 2.6.32-696.30.1.el6.x86_64 (ip-172-31-16-82.ap-northeast-1.compute.internal)      07/08/2018      _x86_64_        (1 CPU)

07:55:15 AM dentunusd   file-nr  inode-nr    pty-nr
07:55:15 AM     61349       736      7230         5

[centos@ip-172-31-16-82 ~]$ slabtop -o | grep dentry
 68120  68110  99%    0.19K   3406       20     13624K dentry
[centos@ip-172-31-16-82 ~]$ slabtop -o | grep dentry
 68160  68136  99%    0.19K   3408       20     13632K dentry

[centos@ip-172-31-16-82 ~]$ free
             total       used       free     shared    buffers     cached
Mem:        499996     178300     321696        160      14292      53844
-/+ buffers/cache:     110164     389832
Swap:      1048572       1280    1047292
[centos@ip-172-31-16-82 ~]$ free
             total       used       free     shared    buffers     cached
Mem:        499996     178300     321696        160      14300      53844
-/+ buffers/cache:     110156     389840
Swap:      1048572       1280    1047292

[centos@ip-172-31-16-82 ~]$ sar -v 0

Linux 2.6.32-696.30.1.el6.x86_64 (ip-172-31-16-82.ap-northeast-1.compute.internal) 07/08/2018 _x86_64_ (1 CPU)

07:55:05 AM dentunusd file-nr inode-nr pty-nr

07:55:05 AM 61349 736 7230 5

[centos@ip-172-31-16-82 ~]$ sar -v 0

Linux 2.6.32-696.30.1.el6.x86_64 (ip-172-31-16-82.ap-northeast-1.compute.internal) 07/08/2018 _x86_64_ (1 CPU)

07:55:15 AM dentunusd file-nr inode-nr pty-nr

07:55:15 AM 61349 736 7230 5

[centos@ip-172-31-16-82 ~]$ slabtop -o | grep dentry

68120 68110 99% 0.19K 3406 20 13624K dentry

[centos@ip-172-31-16-82 ~]$ slabtop -o | grep dentry

68160 68136 99% 0.19K 3408 20 13632K dentry

[centos@ip-172-31-16-82 ~]$ free

total used free shared buffers cached

Mem: 499996 178300 321696 160 14292 53844

-/+ buffers/cache: 110164 389832

Swap: 1048572 1280 1047292

[centos@ip-172-31-16-82 ~]$ free

total used free shared buffers cached

Mem: 499996 178300 321696 160 14300 53844

-/+ buffers/cache: 110156 389840

Swap: 1048572 1280 1047292

ほぼ増加しなくなったことがわかる。

nss-softokn-3.14.3-23.3.el6_8 の環境でも効果がありそうだ。

GuzzleHttp

コードを追っていくと、内部でphp curlを使っているようだ

if (!extension_loaded('curl')) {
    // @codeCoverageIgnoreStart
    throw new RuntimeException('The PHP cURL extension must be installed to use Guzzle.');
    // @codeCoverageIgnoreEnd
}

if (!extension_loaded('curl')) {

// @codeCoverageIgnoreStart

throw new RuntimeException('The PHP cURL extension must be installed to use Guzzle.');

// @codeCoverageIgnoreEnd

}

curl と nss

[centos@ip-172-31-16-82 slab-nss]$ php -r "phpinfo();" | grep "^curl" -A24
PHP Warning:  phpinfo(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Command line code on line 1
curl

cURL support => enabled
cURL Information => 7.19.7
Age => 3
Features
AsynchDNS => No
CharConv => No
Debug => No
GSS-Negotiate => Yes
IDN => Yes
IPv6 => Yes
krb4 => No
Largefile => Yes
libz => Yes
NTLM => Yes
SPNEGO => No
SSL => Yes
SSPI => No
Protocols => tftp, ftp, telnet, dict, ldap, ldaps, http, file, https, ftps, scp, sftp
Host => x86_64-redhat-linux-gnu
SSL Version => NSS/3.27.1
ZLib Version => 1.2.3
libSSH Version => libssh2/1.4.2

[centos@ip-172-31-16-82 slab-nss]$ php -r "phpinfo();" | grep "^curl" -A24

PHP Warning: phpinfo(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Command line code on line 1

curl

cURL support => enabled

cURL Information => 7.19.7

Age => 3

Features

AsynchDNS => No

CharConv => No

Debug => No

GSS-Negotiate => Yes

IDN => Yes

IPv6 => Yes

krb4 => No

Largefile => Yes

libz => Yes

NTLM => Yes

SPNEGO => No

SSL => Yes

SSPI => No

Protocols => tftp, ftp, telnet, dict, ldap, ldaps, http, file, https, ftps, scp, sftp

Host => x86_64-redhat-linux-gnu

SSL Version => NSS/3.27.1

ZLib Version => 1.2.3

libSSH Version => libssh2/1.4.2

[centos@ip-172-31-16-82 slab-nss]$ php -r "print_r(curl_version());"
Array
(
    [version_number] => 463623
    [age] => 3
    [features] => 1597
    [ssl_version_number] => 0
    [version] => 7.19.7
    [host] => x86_64-redhat-linux-gnu
    [ssl_version] => NSS/3.27.1
    [libz_version] => 1.2.3
    [protocols] => Array
        (
            [0] => tftp
            [1] => ftp
            [2] => telnet
            [3] => dict
            [4] => ldap
            [5] => ldaps
            [6] => http
            [7] => file
            [8] => https
            [9] => ftps
            [10] => scp
            [11] => sftp
        )

)

[centos@ip-172-31-16-82 slab-nss]$ php -r "print_r(curl_version());"

Array

(

[version_number] => 463623

[age] => 3

[features] => 1597

[ssl_version_number] => 0

[version] => 7.19.7

[host] => x86_64-redhat-linux-gnu

[ssl_version] => NSS/3.27.1

[libz_version] => 1.2.3

[protocols] => Array

(

[0] => tftp

[1] => ftp

[2] => telnet

[3] => dict

[4] => ldap

[5] => ldaps

[6] => http

[7] => file

[8] => https

[9] => ftps

[10] => scp

[11] => sftp

)

NSS/3.27.1 が使われているようだ。

osにinstallされているnss-softknは使われていない？

curlをmakeして確かめてみる

同じバージョンをインストール

#  cd /usr/local/src/
#  curl -O https://curl.haxx.se/download/archeology/curl-7.19.7.tar.gz
#  tar xvzf curl-7.19.7.tar.gz
#  cd curl-7.19.7

#  yum install gcc g++
#  yum install nspr-devel
#  yum install nss-devel

#  ./configure --with-nss --disable-manual
#  make

# cd /usr/local/src/

# curl -O https://curl.haxx.se/download/archeology/curl-7.19.7.tar.gz

# tar xvzf curl-7.19.7.tar.gz

# cd curl-7.19.7

# yum install gcc g++

# yum install nspr-devel

# yum install nss-devel

# ./configure --with-nss --disable-manual

# make

システムのnssが使われた事の確認と効果の確認

[root@ip-172-31-16-82 curl-7.19.7]# rpm -qa | grep nss
nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64
nss-util-3.36.0-1.el6.x86_64
nss-softokn-freebl-devel-3.14.3-23.3.el6_8.x86_64
nss-sysinit-3.36.0-8.el6.x86_64
nss-devel-3.36.0-8.el6.x86_64
openssh-server-5.3p1-123.el6_9.x86_64
nss-util-devel-3.36.0-1.el6.x86_64
nss-softokn-devel-3.14.3-23.3.el6_8.x86_64
nss-3.36.0-8.el6.x86_64
nss-tools-3.36.0-8.el6.x86_64
nss-softokn-3.14.3-23.3.el6_8.x86_64
openssl-1.0.1e-57.el6.x86_64
openssh-5.3p1-123.el6_9.x86_64
openssh-clients-5.3p1-123.el6_9.x86_64

[root@ip-172-31-16-82 curl-7.19.7]# ./src/curl -V
curl 7.19.7 (x86_64-unknown-linux-gnu) libcurl/7.19.7 NSS/3.36
Protocols: tftp ftp telnet dict http file https ftps
Features: IPv6 Largefile SSL

[root@ip-172-31-16-82 curl-7.19.7]# strace -fc -e trace=access ./src/curl  -s https://www.google.co.jp/ > /dev/null
Process 28940 attached
Process 28941 attached
Process 28942 attached
Process 28943 attached
Process 28944 attached
Process 28946 attached
Process 28945 attached
Process 28947 attached
Process 28949 attached
Process 28948 attached
Process 28950 attached
Process 28951 attached
Process 28953 attached
Process 28952 attached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  0.00    0.000000           0       938       926 access
------ ----------- ----------- --------- --------- ----------------
100.00    0.000000                   938       926 total
[root@ip-172-31-16-82 curl-7.19.7]# NSS_SDB_USE_CACHE=yes strace -fc -e trace=access ./src/curl  -s https://www.google.co.jp/ > /dev/null
Process 29010 attached
Process 29011 attached
Process 29012 attached
Process 29014 attached
Process 29015 attached
Process 29017 attached
Process 29016 attached
Process 29018 attached
Process 29020 attached
Process 29019 attached
Process 29021 attached
Process 29022 attached
Process 29024 attached
Process 29023 attached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  0.00    0.000000           0        24        12 access
------ ----------- ----------- --------- --------- ----------------
100.00    0.000000                    24        12 total

[root@ip-172-31-16-82 curl-7.19.7]# rpm -qa | grep nss

nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64

nss-util-3.36.0-1.el6.x86_64

nss-softokn-freebl-devel-3.14.3-23.3.el6_8.x86_64

nss-sysinit-3.36.0-8.el6.x86_64

nss-devel-3.36.0-8.el6.x86_64

openssh-server-5.3p1-123.el6_9.x86_64

nss-util-devel-3.36.0-1.el6.x86_64

nss-softokn-devel-3.14.3-23.3.el6_8.x86_64

nss-3.36.0-8.el6.x86_64

nss-tools-3.36.0-8.el6.x86_64

nss-softokn-3.14.3-23.3.el6_8.x86_64

openssl-1.0.1e-57.el6.x86_64

openssh-5.3p1-123.el6_9.x86_64

openssh-clients-5.3p1-123.el6_9.x86_64

[root@ip-172-31-16-82 curl-7.19.7]# ./src/curl -V

curl 7.19.7 (x86_64-unknown-linux-gnu) libcurl/7.19.7 NSS/3.36

Protocols: tftp ftp telnet dict http file https ftps

Features: IPv6 Largefile SSL

[root@ip-172-31-16-82 curl-7.19.7]# strace -fc -e trace=access ./src/curl -s https://www.google.co.jp/ > /dev/null

Process 28940 attached

Process 28941 attached

Process 28942 attached

Process 28943 attached

Process 28944 attached

Process 28946 attached

Process 28945 attached

Process 28947 attached

Process 28949 attached

Process 28948 attached

Process 28950 attached

Process 28951 attached

Process 28953 attached

Process 28952 attached

% time seconds usecs/call calls errors syscall

------ ----------- ----------- --------- --------- ----------------

0.00 0.000000 0 938 926 access

------ ----------- ----------- --------- --------- ----------------

100.00 0.000000 938 926 total

[root@ip-172-31-16-82 curl-7.19.7]# NSS_SDB_USE_CACHE=yes strace -fc -e trace=access ./src/curl -s https://www.google.co.jp/ > /dev/null

Process 29010 attached

Process 29011 attached

Process 29012 attached

Process 29014 attached

Process 29015 attached

Process 29017 attached

Process 29016 attached

Process 29018 attached

Process 29020 attached

Process 29019 attached

Process 29021 attached

Process 29022 attached

Process 29024 attached

Process 29023 attached

% time seconds usecs/call calls errors syscall

------ ----------- ----------- --------- --------- ----------------

0.00 0.000000 0 24 12 access

------ ----------- ----------- --------- --------- ----------------

100.00 0.000000 24 12 total

nss-softokn-3.14.3-23.3 で nss-softokn-3.14.3-23.3 が使えているだろうと思われる。

CentOS6.6 に wget, epel, pip, aws cli のインストール

コメントを残す

wget

[centos@ip-172-31-16-82 slab-nss]$ sudo yum install wget
Loaded plugins: fastestmirror, presto
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: ftp.iij.ad.jp
 * extras: ftp.iij.ad.jp
 * updates: ftp.iij.ad.jp
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.12-10.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================================================================================================================================================
 Package                                         Arch                                              Version                                                Repository                                       Size
================================================================================================================================================================================================================
Installing:
 wget                                            x86_64                                            1.12-10.el6                                            base                                            484 k

Transaction Summary
================================================================================================================================================================================================================
Install       1 Package(s)

[centos@ip-172-31-16-82 slab-nss]$ sudo yum install wget

Loaded plugins: fastestmirror, presto

Setting up Install Process

Loading mirror speeds from cached hostfile

* base: ftp.iij.ad.jp

* extras: ftp.iij.ad.jp

* updates: ftp.iij.ad.jp

Resolving Dependencies

--> Running transaction check

---> Package wget.x86_64 0:1.12-10.el6 will be installed

--> Finished Dependency Resolution

Dependencies Resolved

================================================================================================================================================================================================================

Package Arch Version Repository Size

Installing:

wget x86_64 1.12-10.el6 base 484 k

Transaction Summary

Install 1 Package(s)

epel

[centos@ip-172-31-16-82 slab-nss]$ wget http://ftp.riken.jp/Linux/fedora/epel/6/x86_64/Packages/e/epel-release-6-8.noarch.rpm
--2018-07-07 08:34:08--  http://ftp.riken.jp/Linux/fedora/epel/6/x86_64/Packages/e/epel-release-6-8.noarch.rpm
Resolving ftp.riken.jp... 134.160.38.1
Connecting to ftp.riken.jp|134.160.38.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14540 (14K) [application/x-rpm]
Saving to: “epel-release-6-8.noarch.rpm”

100%[======================================================================================================================================================================>] 14,540      --.-K/s   in 0.006s

2018-07-07 08:34:08 (2.30 MB/s) - “epel-release-6-8.noarch.rpm” saved [14540/14540]

[centos@ip-172-31-16-82 slab-nss]$ sudo rpm -ivh ./epel-release-6-8.noarch.rpm
warning: ./epel-release-6-8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing...                ########################################### [100%]
   1:epel-release           ########################################### [100%]

[centos@ip-172-31-16-82 slab-nss]$ wget http://ftp.riken.jp/Linux/fedora/epel/6/x86_64/Packages/e/epel-release-6-8.noarch.rpm

--2018-07-07 08:34:08-- http://ftp.riken.jp/Linux/fedora/epel/6/x86_64/Packages/e/epel-release-6-8.noarch.rpm

Resolving ftp.riken.jp... 134.160.38.1

Connecting to ftp.riken.jp|134.160.38.1|:80... connected.

HTTP request sent, awaiting response... 200 OK

Length: 14540 (14K) [application/x-rpm]

Saving to: “epel-release-6-8.noarch.rpm”

100%[======================================================================================================================================================================>] 14,540 --.-K/s in 0.006s

2018-07-07 08:34:08 (2.30 MB/s) - “epel-release-6-8.noarch.rpm” saved [14540/14540]

[centos@ip-172-31-16-82 slab-nss]$ sudo rpm -ivh ./epel-release-6-8.noarch.rpm

warning: ./epel-release-6-8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY

Preparing... ########################################### [100%]

1:epel-release ########################################### [100%]

CentOS : EPEL リポジトリ追加

php5.5を入れるために

[centos@ip-172-31-16-82 slab-nss]$ sudo rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
Retrieving http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
warning: /var/tmp/rpm-tmp.9t2Bg0: Header V4 DSA/SHA1 Signature, key ID 00f97f56: NOKEY
Preparing...                ########################################### [100%]
   1:remi-release           ########################################### [100%]

[centos@ip-172-31-16-82 slab-nss]$ sudo rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-6.rpm

Retrieving http://rpms.famillecollet.com/enterprise/remi-release-6.rpm

warning: /var/tmp/rpm-tmp.9t2Bg0: Header V4 DSA/SHA1 Signature, key ID 00f97f56: NOKEY

Preparing... ########################################### [100%]

1:remi-release ########################################### [100%]

pip

[centos@ip-172-31-16-82 slab-nss]$ sudo yum install --enablerepo=epel python-pip
Loaded plugins: fastestmirror, presto
Setting up Install Process
Loading mirror speeds from cached hostfile
epel/metalink                                                                                                                                                                            | 7.7 kB     00:00
 * base: ftp.iij.ad.jp
 * epel: ftp.iij.ad.jp
 * extras: ftp.iij.ad.jp
 * updates: ftp.iij.ad.jp
epel                                                                                                                                                                                     | 3.2 kB     00:00
epel/primary                                                                                                                                                                             | 3.2 MB     00:00
epel                                                                                                                                                                                                12515/12515
Resolving Dependencies
--> Running transaction check
---> Package python-pip.noarch 0:7.1.0-1.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================================================================================================================================================
 Package                                             Arch                                            Version                                                Repository                                     Size
================================================================================================================================================================================================================
Installing:
 python-pip                                          noarch                                          7.1.0-1.el6                                            epel                                          1.5 M

Transaction Summary
================================================================================================================================================================================================================
Install       1 Package(s)

Total download size: 1.5 M
Installed size: 6.6 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
epel/prestodelta                                                                                                                                                                         |  770 B     00:00
Processing delta metadata
Package(s) data still to download: 1.5 M
python-pip-7.1.0-1.el6.noarch.rpm                                                                                                                                                        | 1.5 MB     00:00
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
Importing GPG key 0x0608B895:
 Userid : EPEL (6) <epel@fedoraproject.org>
 Package: epel-release-6-8.noarch (installed)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing : python-pip-7.1.0-1.el6.noarch                                                                                                                                                                1/1
  Verifying  : python-pip-7.1.0-1.el6.noarch                                                                                                                                                                1/1

Installed:
  python-pip.noarch 0:7.1.0-1.el6

Complete!

[centos@ip-172-31-16-82 slab-nss]$ sudo yum install --enablerepo=epel python-pip

Loaded plugins: fastestmirror, presto

Setting up Install Process

Loading mirror speeds from cached hostfile

epel/metalink | 7.7 kB 00:00

* base: ftp.iij.ad.jp

* epel: ftp.iij.ad.jp

* extras: ftp.iij.ad.jp

* updates: ftp.iij.ad.jp

epel | 3.2 kB 00:00

epel/primary | 3.2 MB 00:00

epel 12515/12515

Resolving Dependencies

--> Running transaction check

---> Package python-pip.noarch 0:7.1.0-1.el6 will be installed

--> Finished Dependency Resolution

Dependencies Resolved

Package Arch Version Repository Size

Installing:

python-pip noarch 7.1.0-1.el6 epel 1.5 M

Transaction Summary

Install 1 Package(s)

Total download size: 1.5 M

Installed size: 6.6 M

Is this ok [y/N]: y

Downloading Packages:

Setting up and reading Presto delta metadata

epel/prestodelta | 770 B 00:00

Processing delta metadata

Package(s) data still to download: 1.5 M

python-pip-7.1.0-1.el6.noarch.rpm | 1.5 MB 00:00

warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY

Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

Importing GPG key 0x0608B895:

Userid : EPEL (6) <epel@fedoraproject.org>

Package: epel-release-6-8.noarch (installed)

From : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

Is this ok [y/N]: y

Running rpm_check_debug

Running Transaction Test

Transaction Test Succeeded

Running Transaction

Warning: RPMDB altered outside of yum.

Installing : python-pip-7.1.0-1.el6.noarch 1/1

Verifying : python-pip-7.1.0-1.el6.noarch 1/1

Installed:

python-pip.noarch 0:7.1.0-1.el6

Complete!

CentOS 6 の python 2.6.6 に pip を導入する手順

aws cli

[centos@ip-172-31-16-82 slab-nss]$ sudo pip install awscli
・・・
Successfully installed awscli-1.15.53 botocore-1.10.52 colorama-0.3.9 docutils-0.14 futures-3.2.0 jmespath-0.9.3 ordereddict-1.1 pyasn1-0.4.3 python-dateutil-2.6.1 rsa-3.4.2 s3transfer-0.1.13 simplejson-3.3.0

[centos@ip-172-31-16-82 slab-nss]$ sudo pip install awscli

・・・

Successfully installed awscli-1.15.53 botocore-1.10.52 colorama-0.3.9 docutils-0.14 futures-3.2.0 jmespath-0.9.3 ordereddict-1.1 pyasn1-0.4.3 python-dateutil-2.6.1 rsa-3.4.2 s3transfer-0.1.13 simplejson-3.3.0

https://docs.aws.amazon.com/ja_jp/cli/latest/userguide/awscli-install-linux.html

php5.5

[centos@ip-172-31-16-82 slab-nss]$ sudo yum install --enablerepo=remi --enablerepo=remi-php55 php
Loaded plugins: fastestmirror, presto
Setting up Install Process
Loading mirror speeds from cached hostfile
epel/metalink
 * base: ftp.iij.ad.jp
 * epel: ftp.iij.ad.jp
 * extras: ftp.iij.ad.jp
 * remi: ftp.riken.jp
 * remi-php55: ftp.riken.jp
 * remi-safe: ftp.riken.jp
 * updates: ftp.iij.ad.jp
base
extras
remi
remi/primary_db
remi-php55
remi-php55/primary_db
remi-safe
remi-safe/primary_db
updates
Resolving Dependencies
--> Running transaction check
---> Package php.x86_64 0:5.3.3-49.el6 will be updated
---> Package php.x86_64 0:5.5.38-8.el6.remi will be an update
--> Processing Dependency: php-common(x86-64) = 5.5.38-8.el6.remi for package: php-5.5.38-8.el6.remi.x86_64
--> Processing Dependency: php-cli(x86-64) = 5.5.38-8.el6.remi for package: php-5.5.38-8.el6.remi.x86_64
--> Running transaction check
---> Package php-cli.x86_64 0:5.3.3-49.el6 will be updated
---> Package php-cli.x86_64 0:5.5.38-8.el6.remi will be an update
---> Package php-common.x86_64 0:5.3.3-49.el6 will be updated
---> Package php-common.x86_64 0:5.5.38-8.el6.remi will be an update
--> Processing Dependency: php-pecl-zip(x86-64) for package: php-common-5.5.38-8.el6.remi.x86_64
--> Processing Dependency: php-pecl-jsonc(x86-64) for package: php-common-5.5.38-8.el6.remi.x86_64
--> Running transaction check
---> Package php-pecl-jsonc.x86_64 0:1.3.10-2.el6.remi.5.5 will be installed
---> Package php-pecl-zip.x86_64 0:1.15.3-1.el6.remi.5.5 will be installed
--> Processing Dependency: libzip5(x86-64) >= 1.5.1 for package: php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64
--> Processing Dependency: libzip.so.5()(64bit) for package: php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64
--> Running transaction check
---> Package libzip5.x86_64 0:1.5.1-1.el6.remi will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================================================
 Package                                            Arch                                       Version
=======================================================================================================================================================
Updating:
 php                                                x86_64                                     5.5.38-8.el6.remi
Installing for dependencies:
 libzip5                                            x86_64                                     1.5.1-1.el6.remi
 php-pecl-jsonc                                     x86_64                                     1.3.10-2.el6.remi.5.5
 php-pecl-zip                                       x86_64                                     1.15.3-1.el6.remi.5.5
Updating for dependencies:
 php-cli                                            x86_64                                     5.5.38-8.el6.remi
 php-common                                         x86_64                                     5.5.38-8.el6.remi

Transaction Summary
=======================================================================================================================================================
Install       3 Package(s)
Upgrade       3 Package(s)

Total download size: 7.8 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
Processing delta metadata
Package(s) data still to download: 7.8 M
(1/6): libzip5-1.5.1-1.el6.remi.x86_64.rpm
(2/6): php-5.5.38-8.el6.remi.x86_64.rpm
(3/6): php-cli-5.5.38-8.el6.remi.x86_64.rpm
(4/6): php-common-5.5.38-8.el6.remi.x86_64.rpm
(5/6): php-pecl-jsonc-1.3.10-2.el6.remi.5.5.x86_64.rpm
(6/6): php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64.rpm
-------------------------------------------------------------------------------------------------------------------------------------------------------
Total
warning: rpmts_HdrFromFdno: Header V4 DSA/SHA1 Signature, key ID 00f97f56: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-remi
Importing GPG key 0x00F97F56:
 Userid : Remi Collet <RPMS@FamilleCollet.com>
 Package: remi-release-6.9-3.el6.remi.noarch (installed)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-remi
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing : libzip5-1.5.1-1.el6.remi.x86_64
  Installing : php-pecl-jsonc-1.3.10-2.el6.remi.5.5.x86_64

WARNING : These php-* RPMs are not official Fedora / Red Hat build and
overrides the official ones. Don't file bugs on Fedora Project nor Red Hat.

Use dedicated forum at http://forum.remirepo.net/

  Updating   : php-common-5.5.38-8.el6.remi.x86_64
  Installing : php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64
  Updating   : php-cli-5.5.38-8.el6.remi.x86_64
  Updating   : php-5.5.38-8.el6.remi.x86_64
  Cleanup    : php-5.3.3-49.el6.x86_64
  Cleanup    : php-cli-5.3.3-49.el6.x86_64
  Cleanup    : php-common-5.3.3-49.el6.x86_64
==========================================================================

 WARNING : PHP 5.5 have reached its "End of Life" in July 2016.
 Even, if this package includes some security fix, backported from 5.6,
 The UPGRADE to a maintained version is very strongly RECOMMENDED.

==========================================================================
  Verifying  : libzip5-1.5.1-1.el6.remi.x86_64
  Verifying  : php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64
  Verifying  : php-cli-5.5.38-8.el6.remi.x86_64
  Verifying  : php-pecl-jsonc-1.3.10-2.el6.remi.5.5.x86_64
  Verifying  : php-common-5.5.38-8.el6.remi.x86_64
  Verifying  : php-5.5.38-8.el6.remi.x86_64
  Verifying  : php-cli-5.3.3-49.el6.x86_64
  Verifying  : php-5.3.3-49.el6.x86_64
  Verifying  : php-common-5.3.3-49.el6.x86_64

Dependency Installed:
  libzip5.x86_64 0:1.5.1-1.el6.remi                            php-pecl-jsonc.x86_64 0:1.3.10-2.el6.remi.5.5                            php-pecl-zip.x8

Updated:
  php.x86_64 0:5.5.38-8.el6.remi

Dependency Updated:
  php-cli.x86_64 0:5.5.38-8.el6.remi                                                                    php-common.x86_64 0:5.5.38-8.el6.remi

Complete!

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

[centos@ip-172-31-16-82 slab-nss]$ sudo yum install --enablerepo=remi --enablerepo=remi-php55 php

Loaded plugins: fastestmirror, presto

Setting up Install Process

Loading mirror speeds from cached hostfile

epel/metalink

* base: ftp.iij.ad.jp

* epel: ftp.iij.ad.jp

* extras: ftp.iij.ad.jp

* remi: ftp.riken.jp

* remi-php55: ftp.riken.jp

* remi-safe: ftp.riken.jp

* updates: ftp.iij.ad.jp

base

extras

remi

remi/primary_db

remi-php55

remi-php55/primary_db

remi-safe

remi-safe/primary_db

updates

Resolving Dependencies

--> Running transaction check

---> Package php.x86_64 0:5.3.3-49.el6 will be updated

---> Package php.x86_64 0:5.5.38-8.el6.remi will be an update

--> Processing Dependency: php-common(x86-64) = 5.5.38-8.el6.remi for package: php-5.5.38-8.el6.remi.x86_64

--> Processing Dependency: php-cli(x86-64) = 5.5.38-8.el6.remi for package: php-5.5.38-8.el6.remi.x86_64

--> Running transaction check

---> Package php-cli.x86_64 0:5.3.3-49.el6 will be updated

---> Package php-cli.x86_64 0:5.5.38-8.el6.remi will be an update

---> Package php-common.x86_64 0:5.3.3-49.el6 will be updated

---> Package php-common.x86_64 0:5.5.38-8.el6.remi will be an update

--> Processing Dependency: php-pecl-zip(x86-64) for package: php-common-5.5.38-8.el6.remi.x86_64

--> Processing Dependency: php-pecl-jsonc(x86-64) for package: php-common-5.5.38-8.el6.remi.x86_64

--> Running transaction check

---> Package php-pecl-jsonc.x86_64 0:1.3.10-2.el6.remi.5.5 will be installed

---> Package php-pecl-zip.x86_64 0:1.15.3-1.el6.remi.5.5 will be installed

--> Processing Dependency: libzip5(x86-64) >= 1.5.1 for package: php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64

--> Processing Dependency: libzip.so.5()(64bit) for package: php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64

--> Running transaction check

---> Package libzip5.x86_64 0:1.5.1-1.el6.remi will be installed

--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================================================

Package Arch Version

=======================================================================================================================================================

Updating:

php x86_64 5.5.38-8.el6.remi

Installing for dependencies:

libzip5 x86_64 1.5.1-1.el6.remi

php-pecl-jsonc x86_64 1.3.10-2.el6.remi.5.5

php-pecl-zip x86_64 1.15.3-1.el6.remi.5.5

Updating for dependencies:

php-cli x86_64 5.5.38-8.el6.remi

php-common x86_64 5.5.38-8.el6.remi

Transaction Summary

=======================================================================================================================================================

Install 3 Package(s)

Upgrade 3 Package(s)

Total download size: 7.8 M

Is this ok [y/N]: y

Downloading Packages:

Setting up and reading Presto delta metadata

Processing delta metadata

Package(s) data still to download: 7.8 M

(1/6): libzip5-1.5.1-1.el6.remi.x86_64.rpm

(2/6): php-5.5.38-8.el6.remi.x86_64.rpm

(3/6): php-cli-5.5.38-8.el6.remi.x86_64.rpm

(4/6): php-common-5.5.38-8.el6.remi.x86_64.rpm

(5/6): php-pecl-jsonc-1.3.10-2.el6.remi.5.5.x86_64.rpm

(6/6): php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64.rpm

-------------------------------------------------------------------------------------------------------------------------------------------------------

Total

warning: rpmts_HdrFromFdno: Header V4 DSA/SHA1 Signature, key ID 00f97f56: NOKEY

Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-remi

Importing GPG key 0x00F97F56:

Userid : Remi Collet <RPMS@FamilleCollet.com>

Package: remi-release-6.9-3.el6.remi.noarch (installed)

From : /etc/pki/rpm-gpg/RPM-GPG-KEY-remi

Is this ok [y/N]: y

Running rpm_check_debug

Running Transaction Test

Transaction Test Succeeded

Running Transaction

Warning: RPMDB altered outside of yum.

Installing : libzip5-1.5.1-1.el6.remi.x86_64

Installing : php-pecl-jsonc-1.3.10-2.el6.remi.5.5.x86_64

WARNING : These php-* RPMs are not official Fedora / Red Hat build and

overrides the official ones. Don't file bugs on Fedora Project nor Red Hat.

Use dedicated forum at http://forum.remirepo.net/

Updating : php-common-5.5.38-8.el6.remi.x86_64

Installing : php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64

Updating : php-cli-5.5.38-8.el6.remi.x86_64

Updating : php-5.5.38-8.el6.remi.x86_64

Cleanup : php-5.3.3-49.el6.x86_64

Cleanup : php-cli-5.3.3-49.el6.x86_64

Cleanup : php-common-5.3.3-49.el6.x86_64

==========================================================================

WARNING : PHP 5.5 have reached its "End of Life" in July 2016.

Even, if this package includes some security fix, backported from 5.6,

The UPGRADE to a maintained version is very strongly RECOMMENDED.

==========================================================================

Verifying : libzip5-1.5.1-1.el6.remi.x86_64

Verifying : php-pecl-zip-1.15.3-1.el6.remi.5.5.x86_64

Verifying : php-cli-5.5.38-8.el6.remi.x86_64

Verifying : php-pecl-jsonc-1.3.10-2.el6.remi.5.5.x86_64

Verifying : php-common-5.5.38-8.el6.remi.x86_64

Verifying : php-5.5.38-8.el6.remi.x86_64

Verifying : php-cli-5.3.3-49.el6.x86_64

Verifying : php-5.3.3-49.el6.x86_64

Verifying : php-common-5.3.3-49.el6.x86_64

Dependency Installed:

libzip5.x86_64 0:1.5.1-1.el6.remi php-pecl-jsonc.x86_64 0:1.3.10-2.el6.remi.5.5 php-pecl-zip.x8

Updated:

php.x86_64 0:5.5.38-8.el6.remi

Dependency Updated:

php-cli.x86_64 0:5.5.38-8.el6.remi php-common.x86_64 0:5.5.38-8.el6.remi

Complete!

php-xml

[centos@ip-172-31-16-82 slab-nss]$ php publish.php
PHP Fatal error:  Class 'SimpleXMLElement' not found in /home/centos/aws/slab-nss/vendor/guzzle/guzzle/src/Guzzle/Http/Message/Response.php on line 886

1 2	[centos@ip-172-31-16-82 slab-nss]$ php publish.php PHP Fatal error: Class 'SimpleXMLElement' not found in /home/centos/aws/slab-nss/vendor/guzzle/guzzle/src/Guzzle/Http/Message/Response.php on line 886

こんなエラーが出たので、

[centos@ip-172-31-16-82 slab-nss]$ sudo yum install --enablerepo=remi --enablerepo=remi-php55 php-xml
Loaded plugins: fastestmirror, presto
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: ftp.iij.ad.jp
 * epel: mirrors.aliyun.com
 * extras: ftp.iij.ad.jp
 * remi: ftp.riken.jp
 * remi-php55: ftp.riken.jp
 * remi-safe: ftp.riken.jp
 * updates: ftp.iij.ad.jp
Resolving Dependencies
--> Running transaction check
---> Package php-xml.x86_64 0:5.5.38-8.el6.remi will be installed
--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.24)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64
--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.22)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64
--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.18)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64
--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.13)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64
--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.11)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64
--> Processing Dependency: libxslt.so.1()(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64
--> Processing Dependency: libexslt.so.0()(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64
--> Running transaction check
---> Package libxslt.x86_64 0:1.1.26-2.el6_3.1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================================================
 Package                                        Arch                                          Version
=======================================================================================================================================================
Installing:
 php-xml                                        x86_64                                        5.5.38-8.el6.remi
Installing for dependencies:
 libxslt                                        x86_64                                        1.1.26-2.el6_3.1

Transaction Summary
=======================================================================================================================================================
Install       2 Package(s)

Total download size: 670 k
Installed size: 2.9 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
Processing delta metadata
Package(s) data still to download: 670 k
(1/2): libxslt-1.1.26-2.el6_3.1.x86_64.rpm
(2/2): php-xml-5.5.38-8.el6.remi.x86_64.rpm
-------------------------------------------------------------------------------------------------------------------------------------------------------
Total
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : libxslt-1.1.26-2.el6_3.1.x86_64
  Installing : php-xml-5.5.38-8.el6.remi.x86_64
  Verifying  : libxslt-1.1.26-2.el6_3.1.x86_64
  Verifying  : php-xml-5.5.38-8.el6.remi.x86_64

Installed:
  php-xml.x86_64 0:5.5.38-8.el6.remi

Dependency Installed:
  libxslt.x86_64 0:1.1.26-2.el6_3.1

Complete!

[centos@ip-172-31-16-82 slab-nss]$ sudo yum install --enablerepo=remi --enablerepo=remi-php55 php-xml

Loaded plugins: fastestmirror, presto

Setting up Install Process

Loading mirror speeds from cached hostfile

* base: ftp.iij.ad.jp

* epel: mirrors.aliyun.com

* extras: ftp.iij.ad.jp

* remi: ftp.riken.jp

* remi-php55: ftp.riken.jp

* remi-safe: ftp.riken.jp

* updates: ftp.iij.ad.jp

Resolving Dependencies

--> Running transaction check

---> Package php-xml.x86_64 0:5.5.38-8.el6.remi will be installed

--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.24)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64

--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.22)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64

--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.18)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64

--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.13)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64

--> Processing Dependency: libxslt.so.1(LIBXML2_1.0.11)(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64

--> Processing Dependency: libxslt.so.1()(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64

--> Processing Dependency: libexslt.so.0()(64bit) for package: php-xml-5.5.38-8.el6.remi.x86_64

--> Running transaction check

---> Package libxslt.x86_64 0:1.1.26-2.el6_3.1 will be installed

--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================================================

Package Arch Version

=======================================================================================================================================================

Installing:

php-xml x86_64 5.5.38-8.el6.remi

Installing for dependencies:

libxslt x86_64 1.1.26-2.el6_3.1

Transaction Summary

=======================================================================================================================================================

Install 2 Package(s)

Total download size: 670 k

Installed size: 2.9 M

Is this ok [y/N]: y

Downloading Packages:

Setting up and reading Presto delta metadata

Processing delta metadata

Package(s) data still to download: 670 k

(1/2): libxslt-1.1.26-2.el6_3.1.x86_64.rpm

(2/2): php-xml-5.5.38-8.el6.remi.x86_64.rpm

-------------------------------------------------------------------------------------------------------------------------------------------------------

Total

Running rpm_check_debug

Running Transaction Test

Transaction Test Succeeded

Running Transaction

Installing : libxslt-1.1.26-2.el6_3.1.x86_64

Installing : php-xml-5.5.38-8.el6.remi.x86_64

Verifying : libxslt-1.1.26-2.el6_3.1.x86_64

Verifying : php-xml-5.5.38-8.el6.remi.x86_64

Installed:

php-xml.x86_64 0:5.5.38-8.el6.remi

Dependency Installed:

libxslt.x86_64 0:1.1.26-2.el6_3.1

Complete!

AWSのt2.microでAWS SKDのcomposerがこけたので、スワップを作成

コメントを残す

[centos@ip-172-31-16-82 slab-nss]$ composer.phar update
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 3 installs, 0 updates, 0 removals
  - Installing symfony/event-dispatcher (v2.6.13): Downloading (100%)
The following exception is caused by a lack of memory or swap, or not having swap configured
Check https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors for details

PHP Warning:  proc_open(): fork failed - Cannot allocate memory in phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php on line 959

Warning: proc_open(): fork failed - Cannot allocate memory in phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php on line 959

  [ErrorException]
  proc_open(): fork failed - Cannot allocate memory


update [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--lock] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [--with-dependencies] [--with-all-dependencies] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--prefer-stable] [--prefer-lowest] [-i|--interactive] [--root-reqs] [--] [<packages>]...

[centos@ip-172-31-16-82 slab-nss]$ composer.phar update

Loading composer repositories with package information

Updating dependencies (including require-dev)

Package operations: 3 installs, 0 updates, 0 removals

- Installing symfony/event-dispatcher (v2.6.13): Downloading (100%)

The following exception is caused by a lack of memory or swap, or not having swap configured

Check https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors for details

PHP Warning: proc_open(): fork failed - Cannot allocate memory in phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php on line 959

Warning: proc_open(): fork failed - Cannot allocate memory in phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php on line 959

[ErrorException]

proc_open(): fork failed - Cannot allocate memory

update [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--lock] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [--with-dependencies] [--with-all-dependencies] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--prefer-stable] [--prefer-lowest] [-i|--interactive] [--root-reqs] [--] [<packages>]...

となったので、

[root@ip-172-31-16-82 slab-nss]# /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 15.1272 s, 71.0 MB/s
[root@ip-172-31-16-82 slab-nss]# /sbin/mkswap /var/swap.1
Setting up swapspace version 1, size = 1048572 KiB
no label, UUID=001c4bd6-1612-4241-9a0c-8fe3b27f5e08
[root@ip-172-31-16-82 slab-nss]# /sbin/swapon /var/swap.1

[root@ip-172-31-16-82 slab-nss]# /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB) copied, 15.1272 s, 71.0 MB/s

[root@ip-172-31-16-82 slab-nss]# /sbin/mkswap /var/swap.1

Setting up swapspace version 1, size = 1048572 KiB

no label, UUID=001c4bd6-1612-4241-9a0c-8fe3b27f5e08

[root@ip-172-31-16-82 slab-nss]# /sbin/swapon /var/swap.1

と作成

[centos@ip-172-31-16-82 slab-nss]$ composer.phar update
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 3 installs, 0 updates, 0 removals
  - Installing symfony/event-dispatcher (v2.6.13): Loading from cache
  - Installing guzzle/guzzle (v3.9.3): Downloading (100%)
  - Installing aws/aws-sdk-php (2.8.31): Downloading (100%)
symfony/event-dispatcher suggests installing symfony/dependency-injection ()
symfony/event-dispatcher suggests installing symfony/http-kernel ()
guzzle/guzzle suggests installing guzzlehttp/guzzle (Guzzle 5 has moved to a new package name. The package you have installed, Guzzle 3, is deprecated.)
aws/aws-sdk-php suggests installing doctrine/cache (Adds support for caching of credentials and responses)
aws/aws-sdk-php suggests installing ext-apc (Allows service description opcode caching, request and response caching, and credentials caching)
aws/aws-sdk-php suggests installing monolog/monolog (Adds support for logging HTTP requests and responses)
aws/aws-sdk-php suggests installing symfony/yaml (Eases the ability to write manifests for creating jobs in AWS Import/Export)
Package guzzle/guzzle is abandoned, you should avoid using it. Use guzzlehttp/guzzle instead.
Writing lock file
Generating autoload files

[centos@ip-172-31-16-82 slab-nss]$ composer.phar update

Loading composer repositories with package information

Updating dependencies (including require-dev)

Package operations: 3 installs, 0 updates, 0 removals

- Installing symfony/event-dispatcher (v2.6.13): Loading from cache

- Installing guzzle/guzzle (v3.9.3): Downloading (100%)

- Installing aws/aws-sdk-php (2.8.31): Downloading (100%)

symfony/event-dispatcher suggests installing symfony/dependency-injection ()

symfony/event-dispatcher suggests installing symfony/http-kernel ()

guzzle/guzzle suggests installing guzzlehttp/guzzle (Guzzle 5 has moved to a new package name. The package you have installed, Guzzle 3, is deprecated.)

aws/aws-sdk-php suggests installing doctrine/cache (Adds support for caching of credentials and responses)

aws/aws-sdk-php suggests installing ext-apc (Allows service description opcode caching, request and response caching, and credentials caching)

aws/aws-sdk-php suggests installing monolog/monolog (Adds support for logging HTTP requests and responses)

aws/aws-sdk-php suggests installing symfony/yaml (Eases the ability to write manifests for creating jobs in AWS Import/Export)

Package guzzle/guzzle is abandoned, you should avoid using it. Use guzzlehttp/guzzle instead.

Writing lock file

Generating autoload files

とできた。

API Gatewayでカスタムドメイン名

コメントを残す

Certificate ManagerでSSL証明書取得

※DNSでもメールでもどちらでも

メールの方が少し早い

先ほど作った証明書を選択、「API」を選択

Route 53で、ターゲットドメインのALIASでドメインを作成

API Gateway → Lambda

コメントを残す

slackのOutgoing Webhooksから受信する

Proxy統合ではなく、Lambda側のインタフェースを決めてしまう

/raspberrypi/{proxy+} を作成し、POSTのみ作成

/raspberrypi/speak で呼び出し、speakの場合、音声再生要求を投げる

Lambda関数を作成

実装はシンプル

https://github.com/nilesflow/Slack2RaspberryPi

下記等参考に

https://docs.aws.amazon.com/ja_jp/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html

query paramaterのtokenの値の検証のみ

tokenの値はコンソール上の環境変数に設定

Lambdaを登録しておく。

フォーマットは下記の形式

https://docs.aws.amazon.com/ja_jp/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-output-format

Lambdaオーソライザー

ロールはLambda実行権限含むものを作成しておく

クエリパラメータにtokenを設定して認証させたいので、認証でLambdaリクエストオーソライザーを設定。

～検証とURLクエリ文字列パラメータを設定。

リクエストパスのproxyを入力しておく

・作成したLambdaを指定

・プロキシ統合は使わずに、呼び出し側ロジックはAPI Gateway側で吸収

・リクエストパラメータのproxyをマッピング

・Content-Type=application/x-www-form-urlencoded で送信される

・それ以外は厳しめに通過禁止に設定しておく

・マッピングテンプレートは次の通り

## slackのOutgoing Webhooksから下記の形式で送信される
## token=viK8ruCmEfBjd9u84MOEeTWo
## team_id=T0001
## team_domain=例
## channel_id=C2147483705
## channel_name=テスト
## timestamp=1355517523.000005
## user_id=U2147483697
## user_name=スティーブ
## text=googlebot: 身軽なツバメの対気速度はどのくらい？
## trigger_word=googlebot:

## POSTパラメータの分割（A=x&B=y）
#set($aParams = $input.body.split(“&”))

## path変数の分割
#set($aPaths = $input.params(‘proxy’).split(“/”))

## 出力されるJSONデータの生成
{
##pathの次階層をactionとする
“action” : “$aPaths[0]”,

## 必要なPOSTパラメータのみをマッピング
#foreach( $param in $aParams)
#set($hParam = $param.split(“=”))
#if( $hParam[0] == “text”)
“text” : “$util.urlDecode($hParam[1])”,
#end
#if( $hParam[0] == “user_name”)
“user” : “$util.urlDecode($hParam[1])”,
#end
#end

## カンマ制御のためのダミー
“dummy” : null
}

Lambda側で定義される次の変数をマッピングしている

action

text

user

・統合レスポンスは200のみ設定

slackへ応答を返すには、200でレスポンスパラメータを返す必要があるため

マッピングテンプレートは次の通り

## for slack Outgoing Webhooks
## 全て200の単一フォーマットで返却
#set ($inputRoot = $input.path(‘$’))

## Lambdaで正常に処理された場合
#if ($input.path(‘$.statusCode’) > 0)

## 200の場合のフォーマット
#if ($input.path(‘$.statusCode’) == 200)
#set ($body = $util.parseJson($input.path(‘$.body’)))
{
“text” : “$body.message”
}

## それ以外の場合のフォーマット
#else ## 200
#set ($body = $util.parseJson($input.path(‘$.body’)))
{
“text” : “$body.error: $body.error_description”
}
#end ## not 200

## 例外等
#else ## statusCode
{
“text” : “$input.path(‘$.errorMessage’)”
}
#end ## not statusCode

Lambdaの応答の次のケース全てを200として返却する

・ステータスコード200：正常

・ステータスコード200以外：異常（Lambda実装コード中で認識したエラー）

・例外等それ以外：異常（Lambda実装コード中以外のエラーも含む）

・ゲートウェイのレスポンスで、API Gatewayで検出したエラーのレスポンスフォーマットを定義する

これも、SlackのOutgoing Webhooksのフォーマットに合わせておく。

400, 500も200で返すようにしておく。

メソッドレスポンスモデルは不要なのか。

https://docs.aws.amazon.com/ja_jp/apigateway/latest/developerguide/api-gateway-method-settings-method-response.html

統合レスポンス、ゲートウェイのレスポンスでまかなえているようだった。

furoom

カテゴリーアーカイブ:Amazon

RTSP Server gst-rtsp-server on Amazon Linux 2

AWS Batch コンピューティング環境の終了タイミング

AWS Batch　起動AMIのユーザデータ

Amazon ECR

Lambda Python runtime environment

AWS SDK for PHPでのslab増大とNSS_SDB_USE_CACHE対策の効果確認

まとめ

各種情報

AWS-SDKでSNSへpublishするテストコードを準備

実行前後の情報

対策コードを追加

実行前後の情報

GuzzleHttp

curl と nss

curlをmakeして確かめてみる

CentOS6.6 に wget, epel, pip, aws cli のインストール

wget

epel

pip

aws cli

php5.5

php-xml

AWSのt2.microでAWS SKDのcomposerがこけたので、スワップを作成

API Gatewayでカスタムドメイン名

API Gateway → Lambda