La distribuzione dei servizi della piattaforma avviene mediante immagini docker, non ci sono requisiti particolari in merito all'orchestratore utilizzato (docker swarm, kubernetes, nomad, mesos).
Installazione ambiente di Test / Sviluppo
Per una installazione di test su singolo nodo si consiglia di usare il service compose presente nei due repository principali:
Per un ambiente di produzione è indispensabile predisporre un ambiente con adeguati livelli di ridondanza e dimensionamento dei singoli servizi in base alla propria infrastruttura e ai propri requisiti in termini di affidabilità e livelli di servizio da garantire.
Non c'e' una ricetta valida a priori per dimensionare il sistema, si consiglia di partire da una configurazione con 2 repliche di ogni servizio stateless e un dimensionamento molto largo (es: 1GB di ram, 1 vCPU) e poi monitorare il proprio sistema per stabilire i requisiti di ogni singolo container: si potrà verificare facilmente che molti microservizi richiedono pochi Mb di ram.
Elenco dei servizi necessari
Ogni servizio si basa su una immagine docker specifica e necessita di adeguate configurazioni. Le configurazioni e i valori di default delle stesse sono documentate nei singoli repository dei microservizi.
Al fine di dare una corretta configurazione dei vari servizi vengono indicate inoltre:
la dipendenza dalle persistenze usate dalla piattaforma
la necessità di pubblicare un endpoint per quel servizio pubblicamente raggiungibile
Sito istituzionale
Servizio
Descrizione
Docker Image
Pubblico
cms-core
Servizio principale del CMS, consente l'accesso della radazione per la gestione dei contenuti e fornisce l'interfaccia di navigazione per i cittadini
Servizio principale della piattaforma, gestisce pratiche, appuntamenti, offre interfacce di configurazione per gli amministratori e di gestione operativa per i funzionari degli enti. E' inoltre l'interfaccia principale anche per l'esperienza utente.
Esempio di un file di deployment per orchestratore Docker Swarm:
File di esempio
version:"3.8"# vim: autoindent tabstop=2 shiftwidth=2 expandtab softtabstop=2 filetype=yaml fileencoding=utf-8networks:oc-prod-public:# usata per pubblicare serviziexternal:trueoc-prod-internal:# usata per la comunicazione tra i componenti dello stackexternal:truex-deploy:&deploy-snippetendpoint_mode:dnsrrplacement:max_replicas_per_node:2constraints: [ node.labels.apps == true ]replicas:1update_config:parallelism:1order:start-firstdelay:30sfailure_action:rollbackmonitor:10srollback_config:parallelism:1delay:10sfailure_action:pause# default: pause or continuemonitor:10s# default: 5sorder:start-firstrestart_policy:condition:anydelay:15s# default: 5s, between attemptsmax_attempts:50window:20s# time to decide if the restart has been sucessfullresources:limits:cpus:'0.2'memory:512Mreservations:memory:32Mx-sdc-env:&sdc-envENV:prodAPP_ENV:prodDB_DRIVER:pdo_pgsqlDB_HOST:'postgres'DB_PORT:5432DB_NAME:ocDB_USER:ocDB_VERSION:14DB_PASSWORD:'oc'MAILER_URL:'smtp://mailserver:1025'SECRET:change_meDEFAULT_FROM_EMAIL_ADDRESS:no-reply@opencity.itEZ_PASSWORD:change_meOCSDC_SCHEME:httpsOCSDC_HOST:app.localtest.meLOGS_PATH:php://stderrENABLE_MIGRATIONS:'false'ENABLE_INSTANCE_CONFIG:'false'# accesso in scrittura da PHPFORMSERVER_PRIVATE_URL:http://form-server:8000# accesso con cache varnishFORMSERVER_PUBLIC_URL:http://form.localtest.me# accesso readonly per adminFORMSERVER_ADMIN_URL:http://forms-readonly.localtest.meWKHTMLTOPDF_SERVICE:http://gotenberg:3000PHP_FPM_USER:wodbyPHP_FPM_GROUP:wodbyPHP_APCU_ENABLED:1PHP_OPCACHE_ENABLE:1PHP_FPM_CLEAR_ENV:'no'PHP_FPM_PM_MAX_CHILDREN:50PHP_FPM_PM_MAX_REQUESTS:2000PHP_FPM_PM_MAX_SPARE_SERVERS:8PHP_FPM_PM_MIN_SPARE_SERVERS:2PHP_FPM_REQUEST_SLOWLOG_TIMEOUT:4000PHP_FPM_PM_START_SERVERS:5PHP_DISPLAY_ERRORS:'off'PHP_DISPLAY_STARTUP_ERRORS:'off'PHP_ERROR_REPORTING:'E_ERROR'PHP_DATE_TIMEZONE:'Europe/Rome'PHP_SESSION_SAVE_HANDLER:redisPHP_SESSION_SAVE_PATH:'tcp://redis:6379'PHP_SESSION_GC_MAXLIFETIME:2880RECAPTCHA_KEY:recaptcha_keyRECAPTCHA_SECRET:recaptcha_secretEWZ_RECAPTCHA_SITE_KEY:recaptcha_keyEWZ_RECAPTCHA_SECRET:recaptcha_secretFEATURE_NEW_OUTDATED_BROWSER:'true'FEATURE_APPLICATION_DETAIL:'true'FEATURE_CALENDAR_TYPE:'true'#FEATURE_DUE_AMOUNT: 'true'#FEATURE_ANALYTICS: 'true'TOKEN_TTL:864000# 10ggCACHE_MAX_AGE:3600KAFKA_URL:vectorKSQLDB_URL:ksqldb-server:8088SKIP_CACHE_WARMUP:'false'TRUSTED_PROXIES:'10.0.0.0/8,172.16.0.0/12,192.168.0.0/16'DEFAULT_CACHE_REDIS_PROVIDER:'redis://redis:6379'DEFAULT_CACHE_LIFETIME:3600DEFAULT_CACHE_PREFIX_SEED:v-2.27.0REGISTRY_API_URL:https://registry.qa.stanzadelcittadino.itREGISTRY_API_KEY:change_meAPI_USER_PASSWORD:change_me# In assenza di una vera autenticazione basata su SPID si può simulare# un login di un utente con le seguenti variabili d'ambiente:shibb_pat_attribute_codicefiscale:CLNVTR76P01G822Qshibb_pat_attribute_cognome:Coliandroshibb_pat_attribute_nome:Vittorinoshibb_pat_attribute_sesso:Mshibb_pat_attribute_emailaddress:info@comune.bugliano.pi.itshibb_pat_attribute_datanascita:1/9/1976shibb_pat_attribute_luogonascita:Ponsaccoshibb_pat_attribute_provincianascita:PIshibb_pat_attribute_telefono:003912378945shibb_pat_attribute_cellulare:333 444 666 99shibb_pat_attribute_indirizzoresidenza:'Via Gramsci, 1'shibb_pat_attribute_capresidenza:56056shibb_pat_attribute_cittaresidenza:Buglianoshibb_pat_attribute_provinciaresidenza:PIshibb_pat_attribute_statoresidenza:Italiashibb_pat_attribute_spidcode:123456789shibb_pat_attribute_x509certificate_issuerdn:FAKE_issuerdnshibb_pat_attribute_x509certificate_subjectdn:FAKE_subjectdn shibb_pat_attribute_x509certificate_base64: "DQpSZXN1bHQgZ29lcyBoZXJlLi4uDQpCYXNlNjQNCg0KQmFzZTY0IGlzIGEgZ2VuZXJpYyB0ZXJtIGZvciBhIG51bWJlciBvZiBzaW1pbGFyIGVuY29kaW5nIHNjaGVtZXMgdGhhdCBlbmNvZGUgYmluYXJ5IGRhdGEgYnkgdHJlYXRpbmcgaXQgbnVtZXJpY2FsbHkgYW5kIHRyYW5zbGF0aW5nIGl0IGludG8gYSBiYXNlIDY0IHJlcHJlc2VudGF0aW9uLiBUaGUgQmFzZTY0IHRlcm0gb3JpZ2luYXRlcyBmcm9tIGEgc3BlY2lmaWMgTUlNRSBjb250ZW50IHRyYW5zZmVyIGVuY29kaW5nLg=="
shibb_Shib-Session-ID:abc123abc123abc123abc123abc123abc123abc123shibb_Shib-Session-Index:abc123abc123abc123abc123abc123abc123abc123shibb_Shib-Authentication-Instant:2000-01-01T00-00Zx-registry-env:®istry-envDJANGO_DATABASE:postgres:5432:oc_registry:oc:ocDJANGO_SECRET_KEY:change_meDJANGO_SETTINGS_MODULE:application_registry.settings_productionKAFKA_BOOTSTRAP_SERVERS:kafka:9092SENTRY_DSN:configs:sf-config-instances:name:oc-instances-v1file:./config/app/instances.ymlvector-api:name:occ-kafka-http-v1file:./config/kafka/vector.tomlservices:# Symfony-core, esposto solo su rete internaapp:image:registry.gitlab.com/opencity-labs/area-personale/core/app:2.27.0stop_grace_period:45sconfigs: - source:sf-config-instancestarget:/var/www/html/config/instances_prod.ymlnetworks: - oc-prod-internalenvironment:<<:*sdc-envhealthcheck:test: ["CMD","curl","-f","http://localhost:80/health-check"]interval:1m30stimeout:10sretries:6start_period:5sdeploy:<<:*deploy-snippetendpoint_mode:vipreplicas:1resources:limits:cpus:'1'memory:2048Mreservations:memory:1024Mlabels:traefik.enable:'false'# Frontend del symfony-corevarnish:image:registry.gitlab.com/opencontent/varnish:1.2stop_grace_period:45snetworks: - oc-prod-public - oc-prod-internalhealthcheck:disable:truedeploy:<<:*deploy-snippetendpoint_mode:dnsrrreplicas:1resources:limits:cpus:'1'memory:2048Mreservations:memory:1024Mlabels:traefik.enable:'true'traefik.http.services.varnish.loadbalancer.server.port:6081# disabilitato altrimenti quando fallisce backend non sfrutto grace#traefik.http.services.varnish.loadbalancer.healthcheck.path: /health-check# Security headerstraefik.http.middlewares.security.headers.frameDeny:'true'traefik.http.middlewares.security.headers.customFrameOptionsValue:'SAMEORIGIN'traefik.http.middlewares.security.headers.stsSeconds:31536000traefik.http.middlewares.security.headers.stsIncludeSubdomains:'true'traefik.http.middlewares.security.headers.forceStsHeader:'true' #traefik.http.middlewares.security.headers.contentSecurityPolicy: "default-src 'self' data:; img-src 'self' data: blob: c.bing.com www.googletagmanager.com *.openstreetmap.org c.clarity.ms cartodb-basemaps-b.global.ssl.fastly.net cartodb-basemaps-c.global.ssl.fastly.net cartodb-basemaps-a.global.ssl.fastly.nett; style-src 'self' 'unsafe-inline' static.opencityitalia.it satisfy.opencontent.it widget.freshworks.com stackpath.bootstrapcdn.com unpkg.com cdn.datatables.net printjs-4de6.kxcdn.com cdnjs.cloudflare.com fonts.googleapis.com; connect-src 'self' s3.eu-west-1.amazonaws.com api.opencityitalia.it qa-genova-id.boat.opencontent.io dev-genova-id.boat.opencontent.io mappe.genova.opencityitalia.it api.opencontent.it api.qa.stanzadelcittadino.it registry.qa.stanzadelcittadino.it satisfy.boat.opencontent.io static.opencityitalia.it www.giscom.cloud nominatim.openstreetmap.org a.clarity.ms d.clarity.ms k.clarity.ms i.clarity.ms h.clarity.ms widget.freshworks.com fiscalcode.opencontent.it form-qa.stanzadelcittadino.it form-readonly-qa.stanzadelcittadino.it cdn.datatables.net satisfy.opencityitalia.it satisfy.opencontent.it satisfy.hasura.app sdc-analytics-dev-charts-exporter.boat.opencontent.io efil-proxy-qa.boat.opencontent.io pmpay-proxy-qa.boat.opencontent.io iris-proxy-qa.boat.opencontent.io mypay-proxy-qa.boat.opencontent.io www.openstreetmap.org flyimg.opencontent.it; script-src 'self' 'unsafe-inline' 'unsafe-eval' static.opencityitalia.it cdn.announcekit.app widget.freshworks.com www.clarity.ms form-qa.stanzadelcittadino.it cdnjs.cloudflare.com cdn.form.io unpkg.com code.jquery.com www.googletagmanager.com cdn.datatables.net printjs-4de6.kxcdn.com satisfy.opencityitalia.it satisfy.opencontent.it www.recaptcha.net www.gstatic.com; object-src 'none'; font-src 'self' data: satisfy.opencontent.it stackpath.bootstrapcdn.com cdnjs.cloudflare.com unpkg.com fonts.gstatic.com; worker-src 'self'; report-uri https://csp-collector.opencontent.it/csp?env=production&mode=enforce&app=sdc-qa; child-src 'self' www.youtube-nocookie.com announcekit.app"
traefik.http.middlewares.security.headers.contentTypeNosniff:'true' traefik.http.middlewares.security.headers.permissionsPolicy: "geolocation=(self), camera=(self), microphone=(self)"
traefik.http.middlewares.security.headers.referrerPolicy:'strict-origin-when-cross-origin'traefik.http.middlewares.security.headers.browserXssFilter:'true'# Cors generali traefik.http.routers.https.rule: 'Method(`GET`, `POST`, `PUT`, `DELETE`, `PATCH`, `HEAD`, `OPTIONS`, `PURGE`) && Host(`app.localtest.me`)'
traefik.http.routers.https.tls:'true'traefik.http.routers.https.tls.certresolver:'myhttpchallenge'traefik.http.routers.https.middlewares:'cors, general-rl, security'# Cors per condivisione autenticazione traefik.http.routers.origin-https.rule: 'Method(`GET`, `PUT`, `DELETE`, `PATCH`, `HEAD`, `PURGE`) && Host(`app.localtest.me`) && PathPrefix(`/lang/api/session-auth`)'
traefik.http.routers.origin-https.tls:'true'traefik.http.routers.origin-https.priority:10001traefik.http.routers.origin-https.tls.certresolver:'myhttpchallenge'traefik.http.routers.origin-https.middlewares:'cors-origin, general-rl, security'traefik.http.middlewares.cors.headers.accesscontrolallowcredentials:'true' traefik.http.middlewares.cors.headers.accesscontrolallowheaders: 'Authorization,Origin,Content-Type,Accept,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-credentials,Referer,X-Hasura-Role,X-Hasura-Admin-Secret,X-Hasura-Allowed-Roles,X-Hasura-Default-Role,X-Hasura-User-Id,X-Hasura-Org-Id,X-Locale'
traefik.http.middlewares.cors.headers.accesscontrolallowmethods:'GET,POST,HEAD,PUT,DELETE,PATCH,OPTIONS'traefik.http.middlewares.cors.headers.accesscontrolalloworiginlist:'*'traefik.http.middlewares.cors.headers.accesscontrolmaxage:100traefik.http.middlewares.cors.headers.addvaryheader:'true'traefik.http.middlewares.cors-origin.headers.accesscontrolallowcredentials:'true' traefik.http.middlewares.cors-origin.headers.accesscontrolallowheaders: 'Authorization,Origin,Content-Type,Accept,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-credentials,Referer,X-Hasura-Role,X-Hasura-Admin-Secret,X-Hasura-Allowed-Roles,X-Hasura-Default-Role,X-Hasura-User-Id,X-Hasura-Org-Id,X-Locale'
traefik.http.middlewares.cors-origin.headers.accesscontrolallowmethods: 'GET,POST,HEAD,PUT,DELETE,PATCH,OPTIONS'
traefik.http.middlewares.cors-origin.headers.accesscontrolmaxage:100traefik.http.middlewares.cors-origin.headers.addvaryheader:'true'environment:# https://github.com/wodby/varnish#environment-variablesVARNISH_BACKEND_GRACE:'2880m'VARNISH_BACKEND_HOST:'app'VARNISH_BACKEND_PORT:'80'VARNISH_BACKEND_PROBE:'/health-check'VARNISH_KEEP_ALL_PARAMS:1VARNISH_CACHE_STATIC_FILES:1##VARNISH_MOBILE_SEPARATE_CASH: 1#VARNISH_BIG_FILES_SIZE: 10485760VARNISHD_MEMORY_SIZE:'1024m'VARNISH_BACKEND_BETWEEN_BYTES_TIMEOUT:30sVARNISH_BACKEND_FIRST_BYTE_TIMEOUT:30sVARNISH_STRIP_COOKIES:'cookies_consent|_[_a-z]+|wooTracker|VCKEY-[a-zA-Z0-9-_]+'VARNISHD_PARAM_WORKSPACE_BACKEND:'256k'VARNISHD_PARAM_WORKSPACE_CLIENT:'256k'VARNISH_SECRET:'change_me'# Manager symfony, non è esposto su rete pubblica esegue migrazioni e script all'avviomanager:image:registry.gitlab.com/opencity-labs/area-personale/core/app:2.27.0stop_grace_period:45sconfigs: - source:sf-config-instancestarget:/var/www/html/config/instances_prod.ymlnetworks: - oc-prod-internalenvironment:<<:*sdc-envDB_USER:oc# Inserire username utente manager se configuratoDB_PASSWORD:'oc'# Inserire password utente manager se configuratoENABLE_MIGRATIONS:'true'deploy:<<:*deploy-snippetendpoint_mode:vipreplicas:1resources:limits:cpus:'1'memory:1024Mreservations:memory:512M# Ci sono 3 valori da tenere sempre uguali: WORKER_NUM, replicas e swam_cronjob_replicasworker:image:registry.gitlab.com/opencity-labs/area-personale/core/app:2.27.0networks: - oc-prod-internalstop_signal:SIGQUITstop_grace_period:20mcommand: [ "./bin/worker-daemon.sh" ]configs: - source:sf-config-instancestarget:/var/www/html/config/instances_prod.ymlvolumes: - ./config/worker/worker-daemon.sh:/var/www/html/bin/worker-daemon.sh:rohealthcheck:disable:trueenvironment:<<:*sdc-envSWARM_SERVICE:"worker"SWARM_STACK:"oc-prod"#DEBUG: 1WORKER_ID:1WORKER_NUM:1MAX_EXECUTIONS:100MAX_ACTIONS:50deploy:<<:*deploy-snippetendpoint_mode:dnsrrreplicas:1update_config:order:stop-firstrestart_policy:condition:noneresources:limits:cpus:'0.5'memory:2048Mreservations:memory:512Mlabels:swarm.cronjob.enable:"true"swarm.cronjob.schedule:"*/3 * * * *"swarm.cronjob.skip-running:"true"swarm.cronjob.replicas:1# Espone delle api rest per la creazione di eventi su kafkavector:image:timberio/vector:0.18.1-debianconfigs: - source:vector-apitarget:/etc/vector/vector.tomlhealthcheck:test: [ "CMD","curl","-f","http://localhost:80/health" ]interval:1m30stimeout:10sretries:6start_period:5senvironment:VECTOR_LOG:debugnetworks: - oc-prod-internaldeploy:<<:*deploy-snippetlabels:traefik.enable:'false'# Microservizio per la creazione di pdfgotenberg:image:gotenberg/gotenberg:7.9.2networks: - oc-prod-internal - oc-prod-publichealthcheck:test: ["CMD","curl","-f","http://localhost:3000/health"]interval:1m30stimeout:10sretries:6start_period:5sdeploy:<<:*deploy-snippetlabels:prometheus.disable:'true'traefik.enable:'true'traefik.http.services.gotenberg.loadbalancer.server.port:3000traefik.http.routers.gotenberg.rule:'Host(`gotenberg.localtest.me`)'traefik.http.routers.gotenberg.tls:'true'traefik.http.routers.gotenberg.tls.certresolver:'myhttpchallenge'traefik.http.routers.gotenberg.middlewares:'goodheaders'# Strumento per testare la posta elettronicamailhog:image:mailhog/mailhognetworks: - oc-prod-internaldeploy:<<:*deploy-snippetlabels:traefik.enable:'true'traefik.http.services.mailhog.loadbalancer.server.port:8025traefik.http.routers.mailhog.rule:Host(`mail.localtest.me`)traefik.http.routers.mailhog.tls:'true'traefik.http.routers.mailhog.middlewares:goodheaderstraefik.http.routers.mailhog.tls.certresolver:'myhttpchallenge'############################################################################### FORMS############################################################################### Frontend per il formserverform-varnish:image:registry.gitlab.com/opencontent/varnish:1.2stop_grace_period:45snetworks: - oc-prod-internal - oc-prod-publicdeploy:<<:*deploy-snippetreplicas:1resources:limits:cpus:'0.5'memory:512Mreservations:cpus:'0.1'memory:128Mlabels:prometheus.enable:'true'prometheus.port:9131traefik.enable:'true'traefik.http.services.formvarnish.loadbalancer.server.port:6081traefik.http.services.formvarnish.loadbalancer.healthcheck.path:/.vchealthztraefik.http.routers.formvarnish.rule:'Method(`GET`, `HEAD`, `OPTIONS`) && Host(`form.localtest.me`)'traefik.http.routers.formvarnish.tls:'true'traefik.http.routers.formvarnish.tls.certresolver:'myhttpchallenge'traefik.http.routers.formvarnish.middlewares:'cors'environment:# https://github.com/wodby/varnish#environment-variablesVARNISH_BACKEND_HOST:'form-server'VARNISH_BACKEND_PORT:'8000'VARNISH_BACKEND_GRACE:'2880m'VARNISH_BACKEND_PROBE:'/form/5d7aa15b18fecd734051ae7f?varnish-health'VARNISH_CACHE_STATIC_FILES:1VARNISHD_MEMORY_SIZE:'256m'VARNISH_BACKEND_BETWEEN_BYTES_TIMEOUT:10sVARNISH_BACKEND_FIRST_BYTE_TIMEOUT:30sVARNISH_KEEP_ALL_PARAMS:1VARNISH_STRIP_COOKIES:'cookie_consent|_[_a-z]+|wooTracker|VCKEY-[a-zA-Z0-9-_]+|_ga|_gid|_gat[a-zA-Z0-9-_]+'VARNISH_SECRET:'nvXq3CkK'VARNISHD_PARAM_WORKSPACE_BACKEND:'256k'VARNISHD_PARAM_WORKSPACE_CLIENT:'256k'# Espone api rest per il crud dei formform-server:image:registry.gitlab.com/opencity-labs/area-personale/form-server:1.3.0networks: - oc-prod-public - oc-prod-internalenvironment:DB_URL:mongodb://mongo:27017/formmanagerMAX_AGE:5S_MAX_AGE:60#healthcheck:# test: [ "CMD", "wget", "--spider", "http://localhost:8000/form/5d7aa15b18fecd734051ae7f" ]# start_period: 30s# timeout: 1s# retries: 1# interval: 1mdeploy:<<:*deploy-snippetlabels:traefik.enable:'true'traefik.http.services.formserver.loadbalancer.server.port:8000# Espongo il formserver aperto in scrittura con oc-auth, sotto /api per il form-builder traefik.http.routers.formserver.rule: 'Method(`GET`, `HEAD`, `OPTIONS`, `POST`, `PUT`, `PATCH`, `DELETE`) && Host(`form-builder.localtest.me`) && PathPrefix(`/api/`)'
traefik.http.routers.formserver.tls:'true'traefik.http.routers.formserver.tls.certresolver:'myhttpchallenge'traefik.http.middlewares.strip-forms-api.stripprefix.prefixes:'/api/'traefik.http.routers.formserver.middlewares:'strip-forms-api'# Espongo anche readonly per gli admin da backend (per vedere subito aggiornamenti) traefik.http.routers.formserver-ro.rule: 'Method(`GET`, `POST`, `HEAD`, `OPTIONS`) && Host(`form-readonly.localtest.me`)'
traefik.http.routers.formserver-ro.tls:'true'traefik.http.routers.formserver-ro.tls.certresolver:'myhttpchallenge'traefik.http.routers.formserver-ro.middlewares:'oc-auth, cors'# Microservizio per la gestione dei componenti comuni dei form (nested form)form-builder:image:registry.gitlab.com/opencity-labs/area-personale/formbuilderjs:0-5-2healthcheck:test: [ "NONE" ]networks: - oc-prod-publicdeploy:<<:*deploy-snippetlabels:traefik.enable:'true'traefik.http.services.formbuilder.loadbalancer.server.port:80traefik.http.routers.formbuilder.rule:'Method(`GET`, `HEAD`, `OPTIONS`) && Host(`form-builder.localtest.me`)'traefik.http.routers.formbuilder.tls:'true'traefik.http.routers.formbuilder.tls.certresolver:'myhttpchallenge'traefik.http.routers.formbuilder.middlewares:'oc-auth'############################################################################### FORMS############################################################################### Interroga la tabella PAYMENTS_STATUS su ksqldb e richiede info ai proxy sui pagamenti pendentipayments-poller:image:registry.gitlab.com/opencity-labs/area-personale/payments-poller:1.0.2networks: - oc-prod-internalenvironment:KSQLDB_SERVER:http://ksqldb-server:8088LOOP_TIME:900LOG_LEVEL:DEBUGHEALTHCHECK_ID:88f0d2a5-0326-46a1-9093-42e83c3017c8PYTHONDEBUG:1deploy:<<:*deploy-snippetresources:reservations:memory:1Glimits:cpus:'0.5'memory:4Ghealthcheck:test: [ NONE ]# Legge gli eventi dal topic applications e crea un evento nel topic payments se necessariopayment-dispatcher:image:registry.gitlab.com/opencity-labs/area-personale/payment-dispatcher:1.2.1networks: - oc-prod-internalenvironment:KAFKA_TOPIC_APPLICATIONS:applicationsKAFKA_TOPIC_PAYMENTS:paymentsKAFKA_BOOTSTRAP_SERVERS:kafka:9092KAFKA_CONSUMER_GROUP_PREFIX:payment-dispatcherPROMETHEUS_JOB_NAME:payment_dispatcherAPP_ID:payment-dispatcher:1.2.1KSQLDB_SERVER:http://ksqldb-server:8088LOG_LEVEL:INFOhealthcheck:test: [ NONE ]deploy:<<:*deploy-snippetresources:limits:cpus:'0.5'memory:1G# Gateway proxy di iris (ambiente di test)iris-proxy:image:registry.gitlab.com/opencity-labs/area-personale/iris-payment-proxy:1.3.3stop_grace_period:5mnetworks: - oc-prod-publicenvironment:CANCEL_PAYMENT:"true"KAFKA_TOPIC_NAME:paymentsKAFKA_BOOTSTRAP_SERVERS:kafka:9092KAFKA_GROUP_ID:iris_payment_proxy IRIS_OTF_PAYMENT_URL: https://apistage.regione.toscana.it/C01/ComunicazionePosizioniDebitorieOTF/v3/IdpAllineamentoPendenzeEnteOTF
IRIS_IUV_URL:https://iristest.rete.toscana.it/IdpBillerNdpServices/GenerazioneIUVService IRIS_OUTCOME_URL: https://apistage.regione.toscana.it/C01/InvioNotificaPagamentoEsito/v3/IdpInformativaPagamento.Esito
IRIS_VERIFY_URL:https://apistage.regione.toscana.it/C01/VerificaStatoPagamento/v3/IdpVerificaStatoPagamentoIRIS_NOTICE_URL:https://iristest.rete.toscana.it/IdpBillerNdpServices/GenerazioneAvvisiServiceEXTERNAL_API_URL:"https://iris-proxy.localtest.me"INTERNAL_API_URL:"http://iris-proxy.localtest.me"BASE_PATH_EVENT:"sdc-payments/iris-proxy/payments/"BASE_PATH_CONFIG:"sdc-payments/iris-proxy/tenants/"# STORAGE_TYPE="LOCAL|MINIO|S3"STORAGE_TYPE:"LOCAL"STORAGE_BUCKET_NAME:"payments"APP_ID:"iris-proxy:1.3.3"deploy:<<:*deploy-snippetlabels:traefik.enable:'true'traefik.http.services.iris-proxy-qa.loadbalancer.server.port:8000traefik.http.routers.iris-proxy.entrypoints:'websecure, web' traefik.http.routers.iris-proxy.rule: 'Host(`iris-proxy.localtest.me`) && PathPrefix(`/{op:(notice|online-payment|receipt|landing)}/{id:[^/]+}`, `/{op:(docs|redoc|openapi.json|tenants|services|notify-payment)}`, `/{op:(tenants|services)}/({id:[^/]+}|schema)`) && METHOD(`GET`, `POST`, `PUT` ,`PATCH`, `DELETE`, `HEAD`, `OPTIONS`)'
traefik.http.routers.iris-proxy.tls:'true'traefik.http.routers.iris-proxy.tls.certresolver:'myhttpchallenge'traefik.http.middlewares.payment-proxy-qa-cors.headers.accesscontrolallowcredentials:'true' traefik.http.middlewares.payment-proxy-qa-cors.headers.accesscontrolallowheaders: 'Authorization,Origin,Content-Type,Accept,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-credentials,Referer'
traefik.http.middlewares.payment-proxy-qa-cors.headers.accesscontrolallowmethods: 'GET,POST,HEAD,PUT,DELETE,PATCH,OPTIONS'
# asterisco perché ognuno di essi è acceduto da tutti i domini custom che avranno gli entitraefik.http.middlewares.payment-proxy-qa-cors.headers.accesscontrolalloworiginlist:'*'traefik.http.middlewares.payment-proxy-qa-cors.headers.accesscontrolmaxage:100traefik.http.middlewares.payment-proxy-qa-cors.headers.addvaryheader:'true'traefik.http.routers.iris-proxy.middlewares:'goodheaders, payment-proxy-qa-cors'volumes: - ./var/tenants:/app/payments/sdc-payments/iris-proxy/tenants - ./var/payments:/app/payments/sdc-payments/iris-proxy/payments# Aggiorna lo stato delle pratiche dopo che si sono verificati dei pagamentikrun-process-payments:image:registry.gitlab.com/opencity-labs/area-personale/core/app:2.27.0configs: - source:sf-config-instancestarget:/var/www/html/config/instances_prod.ymlentrypoint:/bin/krunstop_grace_period:300shealthcheck:disable:truevolumes: - ./config/krun/process-payment:/bin/process-payment - ./config/krun/krun:/bin/krunnetworks: - oc-prod-internalenvironment:<<:*sdc-envKAFKA_TOPIC:"payments"KAFKA_SERVER:"kafka:9092"KAFKA_CONSUMER_GROUP:"oc-prod-krun-process-payments"COMMAND:"/bin/process-payment"#DEBUG: 1VERBOSE:1deploy:<<:*deploy-snippetresources:limits:cpus:'0.2'memory:1024Mupdate_config:parallelism:3order:stop-first############################################################################### REGISTRY############################################################################### Espone le api per la configurazione dei vari sistemi di protocollazioneregistry-rest-api:image:registry.gitlab.com/opencity-labs/area-personale/stanzadelcittadino-application-registry:1.10.2networks: - oc-prod-public - oc-prod-internalenvironment:<<:*registry-envDJANGO_LOG_LEVEL:INFOREGISTRY_API_KEY:change_medeploy:<<:*deploy-snippetlabels:traefik.enable:"true"traefik.http.services.registry.loadbalancer.server.port:8000traefik.http.routers.registry.rule:"Host(`registry.localtest.me`)"traefik.http.routers.registry.tls:"true"traefik.http.routers.registry.tls.certresolver:"myhttpchallenge"traefik.http.middlewares.registry-qa-home-redir.redirectregex.regex:'registry.localtest.me/$$'traefik.http.middlewares.registry-qa-home-redir.redirectregex.replacement:"registry.localtest.me/admin" traefik.http.middlewares.registry-redirect-tenants.replacepathregex.regex: '^/(d3|datagraph|dedagroup|halley|halley-cloud|hypersic|infor|insiel|maggioli|pitre|tinn)/v1/tenants/(.*)'
traefik.http.middlewares.registry-redirect-tenants.replacepathregex.replacement:'/api/v1/tenants/' traefik.http.middlewares.registry-redirect-services.replacepathregex.regex: '^/(d3|datagraph|dedagroup|halley|halley-cloud|hypersic|infor|insiel|maggioli|pitre|tinn)/v1/services/'
traefik.http.middlewares.registry-redirect-services.replacepathregex.replacement:'/api/v1/services/$$1/' traefik.http.middlewares.registry-redirect-service.replacepathregex.regex: '^/(d3|datagraph|dedagroup|halley|halley-cloud|hypersic|infor|insiel|maggioli|pitre|tinn)/v1/services/(.+)'
traefik.http.middlewares.registry-redirect-service.replacepathregex.replacement:'/api/v1/services/$$1/$$2/' traefik.http.middlewares.registry-redirect-service-schema.replacepathregex.regex: '^/(d3|datagraph|dedagroup|halley|halley-cloud|hypersic|infor|insiel|maggioli|pitre|tinn)/v1/schema'
traefik.http.middlewares.registry-redirect-service-schema.replacepathregex.replacement: '/api/v1/services-schema/$$1'
traefik.http.middlewares.registry-cors.headers.accesscontrolallowcredentials:'true'traefik.http.middlewares.registry-cors.headers.accesscontrolallowheaders:'*'traefik.http.middlewares.registry-cors.headers.accesscontrolallowmethods:'GET,POST,HEAD,PUT,OPTIONS'traefik.http.middlewares.registry-cors.headers.accesscontrolalloworiginlist:'*'traefik.http.middlewares.registry-cors.headers.accesscontrolmaxage:100traefik.http.middlewares.registry-cors.headers.addvaryheader:'true' traefik.http.routers.registry.middlewares: "goodheaders, registry-qa-home-redir, registry-redirect-tenants, registry-redirect-services, registry-redirect-service, registry-redirect-service-schema, registry-cors"
prometheus.enable:"true"prometheus.port:8000# Gestisce il sistema di retry della protocollazioneregistry-retry-scheduler:image:registry.gitlab.com/opencity-labs/area-personale/stanzadelcittadino-application-registry:1.10.2command:/app/venv/bin/python3.9 manage.py run_retry_schedulernetworks: - oc-prod-internalenvironment:<<:*registry-envENABLE_SENTRY_EVENT_FILTER:"true"deploy:<<:*deploy-snippet############################################################################### PDND##############################################################################pdnd-connector:image:registry.gitlab.com/opencity-labs/area-personale/pdnd-connector:x.x.xnetworks: - oc-prod-internaldeploy:<<:*deploy-snippetenvironment:ENVIRONMENT:app_environmentSENTRY_TOKEN:sentry DSNHTTP_EXTERNAL_BASEPATH:https://api.qa.stanzadelcittadino.it/pdndSDC_PUBLIC_KEY_ENDPOINT:https://core/.well-known/jwks.jsonCACHE_EXPIRATION:5mCACHE_EVICTION:10mSTORAGE_TYPE:s3STORAGE_BUCKET:s3_bucketSTORAGE_BASE_PATH:pdndSTORAGE_LOCAL_PATH:/data/pdndSTORAGE_S3_KEY:S3_keySTORAGE_S3_SECRET:s3_secretSTORAGE_S3_REGION:eu-west-1STORAGE_S3_ENDPOINT:https://s3.eu-west-1.amazonaws.comSTORAGE_S3_SSL:"false"labels:prometheus.enable:'true'prometheus.port:8000traefik.enable:'true'traefik.docker.network:'oc-prod-internal'traefik.http.services.pdnd-connector.loadbalancer.server.port:8000traefik.http.routers.pdnd-connector.entrypoints:'websecure, web' traefik.http.routers.pdnd-connector.rule: 'Host(`api.qa.stanzadelcittadino.it`) && PathPrefix(`/pdnd/{op:(status|metrics|docs|anpr/accertamento-residenza|anpr/stato-famiglia|tenants|keys|tenants/configs|tenants/clients|e-services)}`, `/pdnd/{op:(status|metrics|docs|anpr/accertamento-residenza|anpr/stato-famiglia|tenants|keys|tenants/configs|tenants/clients|e-services)}/({id:[^/]+})`) && METHOD(`GET`, `POST`, `PUT` ,`PATCH`, `DELETE`, `HEAD`, `OPTIONS`)'
traefik.http.routers.pdnd-connector.tls:'true'traefik.http.middlewares.pdnd-connector-stripper.stripprefix.prefixes:'/pdnd/'traefik.http.routers.pdnd-connector.tls.certresolver:'myhttpchallenge'traefik.http.routers.pdnd-connector.middlewares:'goodheaders, cors-free-all, pdnd-connector-stripper'
Prima di procedere all’avvio dei microservizi si devono compiere alcune operazioni:
Creazione del file delle istanze
Va creato un file instances.yml sull'ambiente che ospiterà la piattaforma, questo file dovrà poi essere condiviso con i servizi che ne hanno bisogno (vedi file di esempio di distribuzione).
Il file è così formato:
instances:# dominio dell'sitanza con prefissostanzadelcittadino.localtest.me/comune-di-bugliano:# codice meccanografico dell'entecodice_meccanografico:c_cbug# identificativo dell'istanzaidentifier:comune-di-bugliano# lingue disponibili (it, de, en)app_locales:it|en# tipo di login (da scegliere in base al provider dell'ente)login_route:login_pat
Andrà creato un blocco istanza per ogni ente che si vorrà ospitare sulla piattaforma
Creazione dei database
Singole istanze degli enti (Symfony core)
Il tipo di multi-tenancy implementata in OpenCity Area personale è di singolo stack applicativo con database dedicato per tenant.
Per poter effettuare un’installazione dell’infrastruttura abbiamo quindi bisogno di un singolo database per tenant con le seguenti caratteristiche.
Postgres >= 11 con estensione postgis >= 3
Permessi
Per ridurre i rischi dovuti a errori o a compromissione delle credenziali vengono usati due utenti differenti:
un utente oc_manager che viene utilizzato per creare i database ed eseguire operazione di struttura (creazione, modifica e cancellazione di tabelle, viste, colonne ecc ecc)
un utente oc_user che viene utilizzato dall’applicativo per effettuare operazioni sui dati
Attualmente nella nostra infrastruttura la creazione del database avviene con l'utente oc_manager, a partire da un template preimpostato con i privilegi necessari.
Creazione del template
Come utente postgres si crea il database impostando i privilegi di default che verranno assegnati a tutti gli oggetti creati in seguito.
A postgres verranno dati privilegi ampi, mentre a oc_user verranno dati privilegi minimali:
CREATEDATABASE_ocENCODING='utf8'CONNECTIONLIMIT=50;ALTERDEFAULT PRIVILEGES FOR USER oc_manager INSCHEMA"public"GRANT ALL ON TYPES TO postgres WITHGRANTOPTION;ALTERDEFAULT PRIVILEGES FOR USER oc_manager INSCHEMA"public"GRANT ALL ON FUNCTIONS TO postgres;ALTERDEFAULT PRIVILEGES FOR USER oc_manager INSCHEMA"public"GRANT ALL ON SEQUENCES TO postgres;ALTERDEFAULT PRIVILEGES FOR USER oc_manager INSCHEMA"public"GRANT ALL PRIVILEGES ON TABLES TO postgres;ALTERDEFAULT PRIVILEGES FOR USER oc_manager INSCHEMA"public"GRANT USAGE ON TYPES TO oc_user;ALTERDEFAULT PRIVILEGES FOR USER oc_manager INSCHEMA"public"GRANTEXECUTEON FUNCTIONS TO oc_user;ALTERDEFAULT PRIVILEGES FOR USER oc_manager INSCHEMA"public"GRANT ALL ON SEQUENCES TO oc_user;ALTERDEFAULT PRIVILEGES FOR USER oc_manager INSCHEMA"public"GRANTSELECT,INSERT,UPDATE,DELETEON TABLES TO oc_user;CREATE EXTENSION IFNOTEXISTS postgis;
In seguito la proprietà del database viene assegnata ad oc_manager
ALTERDATABASE _oc OWNERTO oc_manager;ALTERSCHEMA public OWNERTO oc_manager;
Per creare database come utente postgres gli viene dato il ruolo oc_manager
GRANT oc_manager TO postgres;
Creazione di un nuovo database
Per creare il database e impostarne subito come owner l'utente corretto, si esegue come utente postgres
Il sistema di protocollazione ha invece bisogno di un singolo database con le seguenti caratteristiche:
Postgres >= 11
Creazione di un nuovo utente
CREATEUSERregistry_userWITHPASSWORD'password';
Tuning di alcuni parametri di connessione (questi comandi sono opzionali ma suggeriti da django)
ALTERROLE registry_user SET client_encoding TO'utf8';ALTERROLE registry_user SET default_transaction_isolation TO'read committed';ALTERROLE registry_user SET timezone TO'UTC';
Creo il database e lo assegno all'utente
CREATEDATABASEoc_registryENCODING='utf8'CONNECTIONLIMIT=50;GRANT ALL PRIVILEGES ONDATABASE oc_registry TO registry_user;
Configurazione dei proxy di pagamento
Per garantire una corretta comunicazione tra l'area personale e i proxy di pagamento è necessaria configurare appropriatamente gli endpoint sul proxy di riferimento.
Configurazione dei CORS
Il requisito di base affinchè questi endpoint siano correttamente funzionanti è che siano correttamente racchiuse all'interno di un middleware che gestisca i CORS in maniera adeguata. In particolare questo middleware dovrà essere configurato come segue:
L'unico endpoint interno in questo caso è quello per richiedere l'aggiornamento dello stato di un pagamento. Per garantire quindi che questo venga chiamato solo ed esclusivamente dal microservizio di polling, è necessario che questo venga configurato come endpoint raggiungibile solo internamente alla rete di docker. Di seguito un esempio: