Category: Uncategorized

Posted on

kubernetes pods dying

Why are my pods dying?

pods that die:
homlishWeb
jukebox
weather
pods that don't die:
hcr
prometheus & grafana
nginx

Internets seems to blame the logs. My logs seem to indicate a volume mount issue, possibly NFS?

Posted on

kubernetes upgrade

phomlish@a6:~/kubernetes/dashboard$ find /etc/kubernetes/pki/ -type f -name “*.crt” -print|egrep -v ‘ca.crt$’|xargs -L 1 -t -i bash -c ‘openssl x509 -noout -text -in {}|grep After’
xargs: warning: options –max-lines and –replace/-I/-i are mutually exclusive, ignoring previous –max-lines value
bash -c ‘openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-etcd-client.crt|grep After’
Not After : Sep 6 11:43:38 2024 GMT
bash -c ‘openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-kubelet-client.crt|grep After’
Not After : Sep 6 11:43:37 2024 GMT
bash -c ‘openssl x509 -noout -text -in /etc/kubernetes/pki/front-proxy-client.crt|grep After’
Not After : Sep 6 11:43:38 2024 GMT
bash -c ‘openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/peer.crt|grep After’
Not After : Sep 6 11:43:38 2024 GMT
bash -c ‘openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/server.crt|grep After’
Not After : Sep 6 11:43:38 2024 GMT
bash -c ‘openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/healthcheck-client.crt|grep After’
Not After : Sep 6 11:43:38 2024 GMT
bash -c ‘openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver.crt|grep After’
Not After : Sep 6 11:43:37 2024 GMT
phomlish@a6:~/kubernetes/dashboard$

as root:
cd /etc/kubernetes/
mv {admin.conf,controller-manager.conf,kubelet.conf,scheduler.conf} ~/
kubeadm init phase kubeconfig all
reboot

as phomlish:
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

 

phomlish@a6:~/kubernetes/dashboard$ ./genK8sToken.sh
error: failed to create token: Unauthorized

 

Posted on

mythweb

/usr/local/apache2/conf/mythweb.conf
setenv “/nfs/a2/droot//usr/local/lib/mythtv/mythweb-fixes-33

Posted on

tcpdump

on mac
phomlish@i9w 61 % sudo tcpdump –list-interfaces
1.en0 [Up, Running, Wireless, Associated]
2.awdl0 [Up, Running, Wireless, Associated]
3.llw0 [Up, Running, Wireless, Not associated]
4.utun0 [Up, Running]
5.utun1 [Up, Running]
6.utun2 [Up, Running]
7.lo0 [Up, Running, Loopback]
8.en3 [Up, Running, Disconnected]
9.en4 [Up, Running, Disconnected]
10.en1 [Up, Running, Disconnected]
11.en2 [Up, Running, Disconnected]
12.bridge0 [Up, Running, Disconnected]
13.gif0 [none]
14.stf0 [none]
15.XHC1 [none]
16.XHC0 [none]
17.XHC20 [none]
18.VHC128 [none]
19.ap1 [Wireless, Association status unknown]

tcpdump i- en0
tcpdump i- en0

Posted on

mythtv

https://www.mythtv.org/wiki/Installing_MythTV_on_Ubuntu

There are two official ways to download MythWeb. You can either download it
as part of the MythPlugins tarball from http://www.mythtv.org/ or you can
grab it directly from a git checkout, as described at
http://code.mythtv.org/trac

lynx https://www.mythtv.org/download/mythweb/33

A good dmesg
[Sat Jul 22 05:11:40 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:40 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a video Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:47 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] xc5000: Firmware dvb-fe-xc5000c-4.1.30.7.fw loaded and running.
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->au8522 13-0047->au0828a vbi Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:11:52 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:22:27 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:22:31 2023] xc5000: Firmware dvb-fe-xc5000c-4.1.30.7.fw loaded and running.
[Sat Jul 22 05:22:31 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:22:31 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:22:31 2023] au0828: Disabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0
[Sat Jul 22 05:22:31 2023] au0828: Enabled Source: Xceive XC5000->Auvitek AU8522 QAM/8VSB Frontend->Auvitek AU8522 QAM/8VSB Frontend Ret 0

Posted on

install root certificates

# convert pem top crt
openssl pkcs12 -export -in homlish_ca.cert.pem -out homlish_ca.cert.crt -nodes

# linux:
sudo cp /nfs/a0/usr3/homlishCA/certs/homlish_ca.cert.pem /usr/local/share/ca-certificates/
update-ca-certificates
# I also had to bounce docker
sudo systemctl restart docker

I got distracted when doing mac, plesse edit me

Posted on

kubernetes maintenance

I can’t connect to my cluster:

systemctl restart kubelet

kubectl get nodes

NAME STATUS ROLES AGE VERSION
a6 NotReady control-plane 28m v1.25.2

kubectl uncordon a6

phomlish@a6:~/kubernetes$ k get all
The connection to the server 10.11.1.96:6443 was refused – did you specify the right host or port?

 

Where is the etcd certificate?  How to update?
k -n kube-system describe pod/etcd-a6

–peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
–peer-client-cert-auth=true
–peer-key-file=/etc/kubernetes/pki/etcd/peer.key
–peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

root@a6:~# docker info
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Docker Buildx (Docker Inc., v0.9.1-docker)
scan: Docker Scan (Docker Inc., v0.17.0)

Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 48
Server Version: 20.10.18
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: runc
Init Binary: docker-init
containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc version: v1.1.4-0-g5fd4c4d
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.19.0-21-amd64
Operating System: Debian GNU/Linux 10 (buster)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.25GiB
Name: a6
ID: HEER:PWF2:3EJ3:MFWN:J7UP:JZMO:ATRW:62BE:GN4I:LZQC:UGQC:SGQY
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

– name: METALLB_ML_SECRET_KEY
valueFrom:
secretKeyRef:
key: secretkey
name: memberlist

Error: INSTALLATION FAILED: rendered manifests contain a resource that already exists. Unable to continue with install: CustomResourceDefinition “addresspools.metallb.io” in namespace “” exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key “app.kubernetes.io/managed-by”: must be set to “Helm”; annotation validation error: missing key “meta.helm.sh/release-name”: must be set to “metallb”; annotation validation error: missing key “meta.helm.sh/release-namespace”: must be set to “default”

Error: INSTALLATION FAILED: rendered manifests contain a resource that already exists. Unable to continue with install: ValidatingWebhookConfiguration “metallb-webhook-configuration” in namespace “” exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key “app.kubernetes.io/managed-by”: must be set to “Helm”; annotation validation error: missing key “meta.helm.sh/release-name”: must be set to “metallb”; annotation validation error: missing key “meta.helm.sh/release-namespace”: must be set to “default”

Error: INSTALLATION FAILED: rendered manifests contain a resource that already exists. Unable to continue with install: ValidatingWebhookConfiguration “metallb-webhook-configuration” in namespace “” exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key “app.kubernetes.io/managed-by”: must be set to “Helm”; annotation validation error: missing key “meta.helm.sh/release-name”: must be set to “metallb”; annotation validation error: missing key “meta.helm.sh/release-namespace”: must be set to “default”

Continue reading “kubernetes maintenance”

Posted on

ffmpeg to sound card

I never did figure this out. Maybe ffmpeg needs to be compiled w/ devices

ffmpeg –devices
linux:
[NULL @ 0x3171700] Unable to find a suitable output format for ‘–devices’
–devices: Invalid argument

mac:
[NULL @ 0x7ff53c810600] Unable to find a suitable output format for ‘–devices’
–devices: Invalid argument

Posted on

icecast2

IPASS=

using ffmpeg
to a7:
ffmpeg -re -ss 19 -i 13.mp3 -f mp3 icecast://source:$IPASS@jukebox.homlish.net:8000/jukebox.mp3

to kubernetes:
ffmpeg -re -ss 19 -i 13.m4a -acodec libmp3lame -ab 32k -ac 1 icecast://source:$IPASS@jplay.homlish.net:8000/live.mp3

to play in browser

using vlc
play to sound card:
/Applications/VLC.app/Contents/MacOS/VLC \
-Idummy –play-and-exit \
/private/nfs/a0/usr4/music/Grateful\ Dead/1990/RFK\ 7\ 12\ 90\ set\ 1/13\ cassidy.m4a

play to icecast2
/Applications/VLC.app/Contents/MacOS/VLC \
-Idummy –play-and-exit \
13.m4a \
:sout=#duplicate{dst=std{access=shout,mux=ogg,dst=source:$IPASS@jplay.homlish.net:8000/live}}

/usr/local/bin/vlc -vvv udp://@224.1.1.105:2105 –sout ‘#transcode{acodec=vorb,ab=128,channels=2}:duplicate{dst=std{access=shout,mux=ogg,dst=source:123456@172.31.0.14:8000/r29}}’

finding Mac sound device
w01:~ phomlish$ system_profiler -listDataTypes | grep -i Audio
SPAudioDataType
w01:~ phomlish$ system_profiler SPAudioDataType
Audio:

Devices:

Built-in Microphone:

Default Input Device: Yes
Input Channels: 2
Manufacturer: Apple Inc.
Current SampleRate: 44100
Transport: Built-in
Input Source: Internal Microphone

Built-in Output:

Default Output Device: Yes
Default System Output Device: Yes
Manufacturer: Apple Inc.
Output Channels: 2
Current SampleRate: 44100
Transport: Built-in
Output Source: Internal Speakers

system_profiler SPAudioDataType -xml
HUGE OUTPUT
TODO:
how can we get liquidsoap to play silence?