ECS: aws-cli

ECS: aws-cli curl, unzip は入っていること 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 ARG BASE_IMAGE=myimage-cicd-appbase ARG TAG=latest FROM ${BASE_IMAGE}:${TAG} as server ARG USERNAME=ubuntu ARG GROUPNAME=ubuntu ARG PASSWORD=ubuntu ARG UID=1000 ARG GID=1000 ENV APP_BASE=/usr/src/app \ LIB_BASE=/usr/src/lib \ POETRY_VERSION=1.0.10 \ PATH="/root/.poetry/bin:$PATH" RUN mkdir -p /var/run/gunicorn && mkdir -p /storage RUN apt-get update && apt-get install -y sudo RUN groupadd -g ${GID} ${GROUPNAME} && \ useradd -m -s /bin/bash -u ${UID} -g ${GID} -G sudo ${USERNAME} --home-dir ${APP_BASE} && \ echo ${USERNAME}:${PASSWORD} | chpasswd && \ echo "${USERNAME} ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers # aws-cli RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" RUN unzip awscliv2.zip RUN sudo ./aws/install # Application COPY . ${APP_BASE} WORKDIR ${APP_BASE} RUN pip install pip poetry urllib3==1.26.15 -U && \ poetry config virtualenvs.create false && \ poetry install && pip install urllib3==1.26.15 pyOpenSSL -U RUN chown -R ${USERNAME}:${GROUPNAME} ${APP_BASE} RUN chmod 777 /run/gunicorn USER ${USERNAME} CMD ["/usr/src/app/docker/codebuild/entry_web.sh", "/usr/src/app"]

2024年1月7日 · 1 分

Terraform: Import

Terrafrom Import ドメイン認証 1 # env $(cat ../.env|xargs) terraform -chdir=prod import 'module.ses.aws_route53_record.domain_identity["yourdomain.com"]' Z01825913VYOI2NNQZL0A__amazonses.yourdomain.com_TXT S3 1 2 3 export RES=module.s3_backup.aws_s3_bucket.this export BUCKET=mybucket-prod env $(cat ../.env|xargs) terraform -chdir=prod import $RES $BUKET

2024年1月7日 · 1 分

AWS: EFS: バックアップ

EFS バックアップ AWS Backup https://github.com/hdknr/awsform/blob/main/terraform/modules/backups/main.tf AWS DataSync AWS DataSync を使って S3 から EFS へデータを転送する!

2024年1月5日 · 1 分

django-redis: lock

djanog-redis: cache.lock lock https://github.com/jazzband/django-redis/blob/d94a7f9644b96cc37743914fce899cca942c032a/django_redis/cache.py#L177 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 class RedisCache(BaseCache): ... @omit_exception def lock(self, *args, **kwargs): return self.client.lock(*args, **kwargs) def __init__(self, server: str, params: Dict[str, Any]) -> None: ... self._client_cls = options.get( "CLIENT_CLASS", "django_redis.client.DefaultClient" ) self._client_cls = import_string(self._client_cls) self._client = None ... DefaultClient https://github.com/jazzband/django-redis/blob/d94a7f9644b96cc37743914fce899cca942c032a/django_redis/client/default.py#L29 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 from redis import Redis class DefaultClient: def __init__(self, server, params: Dict[str, Any], backend: BaseCache) -> None: ... self._clients: List[Optional[Redis]] = [None] * len(self._server) ... def lock( self, key: KeyT, version: Optional[int] = None, timeout: Optional[float] = None, sleep: float = 0.1, blocking_timeout: Optional[float] = None, client: Optional[Redis] = None, thread_local: bool = True, ): if client is None: client = self.get_client(write=True) key = self.make_key(key, version=version) return client.lock( key, timeout=timeout, sleep=sleep, blocking_timeout=blocking_timeout, thread_local=thread_local, ) def get_client( self, write: bool = True, tried: Optional[List[int]] = None, ) -> Redis: """ Method used for obtain a raw redis client. This function is used by almost all cache backend operations for obtain a native redis client/connection instance. """ index = self.get_next_client_index(write=write, tried=tried) if self._clients[index] is None: self._clients[index] = self.connect(index) return self._clients[index] # type:ignore Redis https://github.com/redis/redis-py/blob/6d77c6d715430c30f22147f8c572659d77380a9f/redis/client.py#L88 lock: https://github.com/redis/redis-py/blob/6d77c6d715430c30f22147f8c572659d77380a9f/redis/client.py#L439C1-L511C10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 class Redis(RedisModuleCommands, CoreCommands, SentinelCommands): def lock( self, name: str, timeout: Optional[float] = None, sleep: float = 0.1, blocking: bool = True, blocking_timeout: Optional[float] = None, lock_class: Union[None, Any] = None, thread_local: bool = True, ): """ Return a new Lock object using key ``name`` that mimics the behavior of threading.Lock. If specified, ``timeout`` indicates a maximum life for the lock. By default, it will remain locked until release() is called. ``sleep`` indicates the amount of time to sleep per loop iteration when the lock is in blocking mode and another client is currently holding the lock. ``blocking`` indicates whether calling ``acquire`` should block until the lock has been acquired or to fail immediately, causing ``acquire`` to return False and the lock not being acquired. Defaults to True. Note this value can be overridden by passing a ``blocking`` argument to ``acquire``. ``blocking_timeout`` indicates the maximum amount of time in seconds to spend trying to acquire the lock. A value of ``None`` indicates continue trying forever. ``blocking_timeout`` can be specified as a float or integer, both representing the number of seconds to wait. ``lock_class`` forces the specified lock implementation. Note that as of redis-py 3.0, the only lock class we implement is ``Lock`` (which is a Lua-based lock). So, it's unlikely you'll need this parameter, unless you have created your own custom lock class. ``thread_local`` indicates whether the lock token is placed in thread-local storage. By default, the token is placed in thread local storage so that a thread only sees its token, not a token set by another thread. Consider the following timeline: time: 0, thread-1 acquires `my-lock`, with a timeout of 5 seconds. thread-1 sets the token to "abc" time: 1, thread-2 blocks trying to acquire `my-lock` using the Lock instance. time: 5, thread-1 has not yet completed. redis expires the lock key. time: 5, thread-2 acquired `my-lock` now that it's available. thread-2 sets the token to "xyz" time: 6, thread-1 finishes its work and calls release(). if the token is *not* stored in thread local storage, then thread-1 would see the token value as "xyz" and would be able to successfully release the thread-2's lock. In some use cases it's necessary to disable thread local storage. For example, if you have code where one thread acquires a lock and passes that lock instance to a worker thread to release later. If thread local storage isn't disabled in this case, the worker thread won't see the token set by the thread that acquired the lock. Our assumption is that these cases aren't common and as such default to using thread local storage.""" if lock_class is None: lock_class = Lock return lock_class( self, name, timeout=timeout, sleep=sleep, blocking=blocking, blocking_timeout=blocking_timeout, thread_local=thread_local, )

2024年1月5日 · 4 分

MySQL: CDC

MySQL: CDC(Change Data Capture) AWS Database Migration Service による Change Data Capture: 前編 AWS Database Migration Service による Change Data Capture: 後編 AWS DatabaseMigrationService での DB 移行構築ハンズオン RDS MySQL AWS DMS のソースとして AWS が管理する MySQL 互換データベースの使用 バイナリログ(binlog_format == ROW) binlog_row_image == Full binlog_checksum == NONE DMS AWS Database Migration Service DMS で AWS RDS へ継続的に移行してみた DMS Endpoint Connection Test Failed with Secret Manager Manage your AWS DMS endpoint credentials with AWS Secrets Manager How to connect to AWS Secrets Manager service within a Virtual Private Cloud AWS Database Migration Service (AWS DMS) の CDC レプリケーションを使ってみた サーバーレス: ...

2023年12月28日 · 2 分

PyCaret

PyCaret: AutoML https://pycaret.org/ PyCaret 入門 Python で機械学習を自動化しよう!【AutoML】 【AutoML】PyCaret で超ラクに機械学習!!

2023年12月25日 · 1 分

Python プロファイリング

cProfile Python プログラムが遅い原因を調べる方法 1 2 python -m cProfile -o /tmp/epm.prof manage.py reports build 7646 snakeviz /tmp/epm.prof 名前 意味 ncalls 呼び出し回数 tottime 関数の処理時間の合計 (呼び出した関数の処理時間は除外) percall ( tottime の隣) tottime を ncalls で割った値。一回の実行にかかる平均時間 cumtime 関数の処理時間の合計 (呼び出した関数の処理時間も含める) percall (sumtime の隣) cumtime を ncalls で割った値。一回の実行にかかる平均時間 filename:lineno(function) ファイル名、行番号、関数名

2023年12月24日 · 1 分

「この接続ではプライバシーが保護されません」

「この接続ではプライバシーが保護されません」 【18 の方法】「この接続ではプライバシーが保護されません」エラーを解決するには SSL TEST https://www.ssllabs.com/ssltest/index.html DNS CAA DNS CAA (Certification Authority Authorization)利用が標準化 Route 53 が CAA レコードに対応しました!

2023年12月13日 · 1 分

devel を削除したたが devel/0 を作れない

git update-ref -d これだけだと devel/2023-12-14 を pull できない git branch -d devel error: cannot lock ref 'refs/remotes/origin/devel/2023-12-14': 'refs/remotes/origin/devel' exists; cannot create 'refs/remotes/origin/devel/2023-12-14' From github.com:spin-dd/taihei-epm-server ! [new branch] devel/2023-12-14 -> origin/devel/2023-12-14 (unable to update local ref) error: some local refs could not be updated; try running 'git remote prune origin' to remove any old, conflicting branches update-ref でクリアされる git update-ref -d refs/remotes/origin/devel

2023年12月13日 · 1 分

半角カタカナ

半角カタカナ 全角・半角カタカナのバリデーションルール設定方法(正規表現)

2023年12月12日 · 1 分