Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • fb6-wp11-devops/webservice
  • maha1056/devops-pipeline
  • ludo8147/webservice
  • s52888/webservice
  • masi9606/webservice
  • kibu5600/webservice
  • s78689/webservice
  • s50860/webservice
  • s92604/devops-webservice
  • s76867/webservice-devops
  • s92274/webservice
  • s80066/webservice
  • masa1998/webservice
  • s91190/app-service
  • s84985/webservice
  • gjahn/webservice-ws-2425
  • s75359/webservice
  • ouch4861/webservice-ws-24-oc
  • s92274/webservice-msws-24
  • ewbo4360/webservice
20 results
Show changes
Commits on Source (26)
.local
artifact.bin
*terraform*
.ssh
kubectl*
---
workflow:
rules:
- if: >-
$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "main"
when: 'always'
- when: 'never'
- if: $CI_PIPELINE_SOURCE == "merge_request_event" || ($CI_PIPELINE_SOURCE == "push" && ($CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "prod"))
when: always
- when: never
variables:
version: 0.0.$CI_PIPELINE_IID
job_trigger-pipeline:
trigger:
project: 'fb6-wp11-devops/webservice-build-and-publish'
stages:
- test
- build
- publish
test_job:
stage: test
image: public.ecr.aws/docker/library/golang:1.21
script:
- go get -t ./...
- go test -race -v ./...
build_job:
stage: build
rules:
- if: $CI_COMMIT_REF_NAME =~ /prod/
when: always
image: public.ecr.aws/docker/library/golang:1.21
script: |
GOARCH=amd64 GOOS=linux CGO_ENABLED=0 go build -o artifact.bin ./*.go
artifacts:
paths:
- artifact.bin
expire_in: 5 min
publish_job:
stage: publish
rules:
- if: $CI_COMMIT_REF_NAME =~ /prod/
when: always
image: docker:latest
services:
- docker:dind
tags:
- docker-privileged
dependencies:
- build_job
script:
- docker build -t $CI_REGISTRY_IMAGE:$version -f Containerfile .
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
- docker push $CI_REGISTRY_IMAGE:$version
after_script:
- docker logout $CI_REGISTRY
# syntax=docker/dockerfile:1
FROM scratch
COPY artifact.bin /artifact.bin
ENV HOST=0.0.0.0
ENV PORT=8080
EXPOSE 8080
CMD ["/artifact.bin"]
Webservice
==========
# Deliverable: Projektkonzept
A Go-based simple web service meant to be the subject of any tutorial
or even used the project work.
## Infrastruktur
Die Produktivumgebung des Webservice wird auf der Google Cloud Platform (GCP) gehostet:
#### Prerequisites:
- **Details zur Infrastruktur**:
- Maschinentyp: e2-small (1 vCPU, 2 GB RAM)
- Region: Berlin (wenn verfügbar, Europa)
- Betriebssystem: Ubuntu 22.04
- Festplatte: 20 GB
- Tags für Firewall: http-server,https-server
* minimal [required version](./go.mod#L3) of the Go toolchain (install via
system package manager or [by hand](https://go.dev/doc/install))
* [optional] [Redis](https://redis.io/docs/install/) to persist state
## Ressourcenmanagement
- **Dynamische Zuweisung**: Ressourcen werden nach Bedarf zugewiesen und nach Nutzung entzogen, um Google Cloud-Credits effizient zu nutzen
- **Skalierung**: Compute Engine-Instanzen werden nach Ressourcenbedarf skaliert
#### State:
## Bereitstellungsprozess
If the database host is not explicitly defined, then the state is ephemeral. For more
information checkout the [configuration code](./configuration/config.go).
- **OpenTofu**: Deklarative Verwaltung der Cloud-Infrastruktur
- **Ansible**: Konfigurationsmanagement und Anwendungsbereitstellung
## Technologiewahl
#### Build:
- **Programmiersprache**: Go
- **Container-Runtime**: Podman
- **CI/CD**: GitLab
1. Install dependencies: `go get -t ./...`
2. Run locally: `go run .`
3. Execute unit tests: `go test -race -v ./...`
4. Build artifact: `go build -o ./artifact.bin ./*.go`
## Umgebungen
To build for another platform, set `GOOS` and `GOARCH`. To yield a static
binary (fully self-contained, no dynamic linking) set `CGO_ENABLED=0`.
To set a version during build time, add the following CLI option
`-ldflags "-X webservice/configuration.version=${VERSION}"`.
For more information, please refer to the [Makefile](./Makefile).
- **Entwicklung**: Lokal mit Vagrant
- **Produktion**: Erweiterte Sicherheitsmaßnahmen und Monitoring
## Dienste <a id="Dienste"></a>
#### Run:
- **Compute Engine**: Hostet VMs für Webservice und Redis-Datenbank
- **Cloud Monitoring**: Überwacht Leistung, bietet Einblicke in Logs, Metriken und Alarme
```bash
HOST=0.0.0.0 PORT=8080 ./artifact.bin
```
## Visuelle Hilfsmittel
- **Architekturdiagramm**: Erstellung mit draw.io
- TODO
#### Interact:
### CI/CD Pipeline
##### Landing page
Die Konfiguration für den Build des Webservices erfolgt in der GitLab CI/CD-Pipeline. Diese unterscheidet die Stages Test, Build und Publish. Bei Merge-Requests oder Pushes auf den `main`- oder `prod`-Branch wird die Pipeline aktiviert.
plain text:
```bash
curl http://localhost:8080
```
- **Test**: Tests mit golang:1.21 Docker-Image
- **Build**: Baut Webservice für linux_amd64-Systeme und speichert Artefakte in der Package Registry
- **Publish**: Veröffentlicht Docker-Container in der Container Registry
HTML:
```bash
curl --header 'Accept: text/html; charset=utf-8' http://localhost:8080
# or just open in a browser
```
Abhängig vom Branch führt die Pipeline Tests aus, erstellt ein Static Binary und veröffentlicht einen Docker-Container, bei welchem die Versionierung auf der Pipeline-ID basiert.
## Persistenzschicht (Datenbank)
##### Health check
Für die Persistenz wird Redis verwendet, welches in einer separaten Compute Engine-Instanz innerhalb der GCP-Umgebung betrieben wird.
```bash
curl http://localhost:8080/health
```
Die Einrichtung und Konfiguration erfolgt über Ansible.
## Lifecycle
##### Server side environment variables
- **DEV: Workstation als GitLab Runner**: Deploy-Job auf der lokalen Workstation wird durch Registrierung der Workstation als GitLab Runner ausgeführt. Der Runner wird im Hochschul-GitLab registriert und bei Pushes auf den `dev`-Branch getriggert.
- **MAIN: Main to Prod**: Änderungen vom Hauptzweig (`main`) werden in die Produktionsumgebung (`prod`) übernommen
- **FEATURES: Feature to Prod**: Feature-Zweige werden in den Hauptzweig gemerged und schließlich in die Produktionsumgebung überführt
- **Pipeline Trigger**: Die Pipeline wird durch Merge-Requests oder Pushes auf `main`, `dev` oder `prod` aktiviert
List environment variables visible by the webservice process if environment
is not `production`.
## Weitere Konfigurationen
```bash
curl http://localhost:8080/env
```
##### State life cycle
URL slug is used as identifier and the body is the actual *data* being stored.
Please note, when writing (add or change) something, `Content-Type` must be set
in the request header.
Write an entry:
```bash
curl \
-X PUT \
--header 'Content-Type: text/plain; charset=utf-8' \
--data 'foo' \
http://localhost:8080/state/bar
```
Find out MIME type and size of an entry:
```bash
curl \
-X HEAD \
http://localhost:8080/state/bar
```
Obtain an entry:
```bash
curl \
-X GET \
http://localhost:8080/state/bar
```
Remove an entry:
```bash
curl \
-X DELETE \
--verbose \
http://localhost:8080/state/bar
```
List all existing entries (returns JSON or plain text, depending on the `Accept` header):
```bash
curl \
-X GET \
--header 'Accept: text/plain' \
http://localhost:8080/states
```
Upload an entire file:
```bash
curl \
-X PUT \
--header 'Content-Type: application/pdf' \
--upload-file ./example.pdf \
http://localhost:8080/state/pdf-doc
```
Download a file:
```bash
curl \
-X GET \
--output ./example-copy.pdf \
http://localhost:8080/state/pdf-doc
```
- **Monitoring**: Die Anwendung wird mithilfe von GCP (siehe [Dienste](#Dienste)) überwacht
- **Sicherheit**: TLS-Terminierung erfolgt durch ein selbstsigniertes Zertifikat in der Entwicklungsumgebung
- **DNS**: In der Entwicklungsumgebung wird die DNS-Auflösung in der `/etc/hosts`-Datei konfiguriert; es soll nachgewiesen werden, dass die Seite über einen Alias per HTTPS verfügbar ist
#kubectl config current-context
#kubectl api-resources
#kubectl get pods
#init
gcloud container clusters create devops24-cluster \
--zone=europe-west3-a \
--machine-type=e2-medium \
--num-nodes=3 \
--enable-autoscaling \
--min-nodes=3 \
--max-nodes=10
gcloud compute firewall-rules create allow-http-https \
--allow tcp:80,tcp:443 \
--description "Allow HTTP and HTTPS traffic" \
--direction INGRESS
#debug
gcloud container clusters describe devops24-cluster --zone=europe-west3-a
#deploy
kubectl apply -f namespace.yaml
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml
kubectl apply -f ingress.yaml
#size up
gcloud container clusters resize devops24-cluster --num-nodes=5 --zone=europe-west3-a
#debug
kubectl get pods --output wide
kubectl get services --output wide
kubectl get ingress --output wide
#hop in
gcloud compute ssh --zone "europe-west3-a" "gke-devops24-cluster-default-pool-9b4d4dcd-55mt" --project "bht-devops24-ss"
gcloud compute ssh --zone "europe-west3-a" "gke-devops24-cluster-default-pool-9b4d4dcd-n2rb" --project "bht-devops24-ss"
gcloud compute ssh --zone "europe-west3-a" "gke-devops24-cluster-default-pool-9b4d4dcd-phf0" --project "bht-devops24-ss"
#replace
kubectl delete pod [POD_NAME]
kubectl get pods --watch
#delete
gcloud container clusters delete devops24-cluster --zone=europe-west3-a
\ No newline at end of file
apiVersion: apps/v1
kind: Deployment
metadata:
name: devops24-webservice-deployment
namespace: devops24-namespace
spec:
replicas: 3
selector:
matchLabels:
app: devops24-webservice
template:
metadata:
labels:
app: devops24-webservice
spec:
securityContext:
runAsNonRoot: true
containers:
- name: devops24-webservice-container
image: registry.bht-berlin.de:443/masi9606/webservice:0.0.28
env:
- name: HOST
value: "0.0.0.0"
- name: PORT
value: "8080"
ports:
- name: http
containerPort: 8080
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "200m"
memory: "256Mi"
securityContext:
allowPrivilegeEscalation: false
runAsUser: 65534
livenessProbe:
tcpSocket:
port: http
readinessProbe:
httpGet:
path: /health
port: http
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: devops24-webservice-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx-test
rules:
- host: devops24-webservice.35.246.253.128.nip.io
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: devops24-webservice-service
port:
number: 80
apiVersion: v1
kind: Namespace
metadata:
name: devops24-namespace
apiVersion: v1
kind: Service
metadata:
name: devops24-webservice-service
namespace: devops24-namespace
spec:
selector:
app: devops24-webservice
ports:
- protocol: TCP
name: http
port: 80
targetPort: 8080
type: ClusterIP
terraform {
required_version = ">= 1.0"
}
provider "google" {
project = var.projectID
zone = var.zone
credentials = file(var.gcpCredentialsFilePath)
}
locals {
sshUserName = "schnarkus"
}
resource "google_compute_network" "the_network" {
name = "the-network"
}
resource "google_compute_firewall" "gate_guardian" {
name = "gate-guardian"
network = google_compute_network.the_network.name
allow {
protocol = "icmp"
}
allow {
protocol = "tcp"
ports = ["22", "8080"] # ssh localhost
}
source_ranges = ["0.0.0.0/0"]
}
resource "google_compute_instance" "schminstance" {
name = "schminstance"
machine_type = "e2-micro"
boot_disk {
initialize_params {
image = data.google_compute_image.image.self_link
}
}
network_interface {
network = google_compute_network.the_network.name
access_config {}
}
metadata = {
ssh-keys = "${local.sshUserName}:${file(var.sshPublicKeyPath)}"
}
}
data "google_compute_image" "image" {
family = "ubuntu-2004-lts"
project = "ubuntu-os-cloud"
}
# get ip and publish
output "instanceIPv4" {
description = "Public IP address of the Google Compute Engine instance"
value = google_compute_instance.schminstance.network_interface[0].access_config[0].nat_ip
}
1272 tofu init
1273 tofu apply
1274 tofu output instanceIPv4
1275 ssh -i ../.ssh/operator -l schnarkus $(tofu output -raw 'instanceIPv4')
1276 scp -i ../.ssh/operator ../artifact.bin schnarkus@$(tofu output -raw 'instanceIPv4'):~/webservice
1277 curl -s http://$(tofu output -raw 'instanceIPv4'):8080
1278 tofu destroy
HOST=0.0.0.0 PORT=8080 ./webservice
\ No newline at end of file
variable "sshPublicKeyPath" {
type = string
description = "Path to the public part of the SSH key pair"
default = "../.ssh/operator.pub"
}
variable "gcpCredentialsFilePath" {
type = string
description = "Path to a GCP credentials file (e.g., service account key)"
default = "~/.gcp/keyfile.json"
}
variable "projectID" {
type = string
description = "ID of a project within GCP"
default = "bht-devops24-ss"
}
variable "zone" {
type = string
description = "GCP zone to deploy resources in"
default = "europe-west3-b"
}