A comprehensive installation and configuration manual that consolidates all the insights from our previous discussions. It walks you through installing, configuring, and managing a Postfix + Dovecot + MySQL + Roundcube mail server on a typical Ubuntu/Debian system. It includes SSL/TLS, SMTP AUTH, MySQL-based virtual users, and Roundcube webmail.

Installation Manual:

Postfix + Dovecot + MySQL + Roundcube Setup

Table of Contents

1. Introduction
2. Prerequisites & Basic System Setup
3. Install Required Packages
4. Configure MySQL for Virtual Domains & Users
5. Configure Postfix
6. Configure Dovecot
7. Configure Roundcube
8. SSL Certificates with Let’s Encrypt (Optional)
9. Testing the Setup
10. Managing Users
11. Enabling Sent Folders & Basic Roundcube Customization
12. Troubleshooting Tips

1. Introduction

This guide describes how to build a mail server that uses:

• Postfix as the SMTP server (handles sending and receiving).
• Dovecot as the IMAP/POP3 server for mail storage and authentication.
• MySQL for storing virtual domain and mailbox information.
• Roundcube as a webmail client.

When completed, you’ll have:

• Encrypted (TLS) IMAP on port 993 and SMTP on port 587 (for sending).
• MySQL-based user accounts (no need for system users).
• A Roundcube webmail interface for sending and receiving mail.

2. Prerequisites & Basic System Setup

1. Ubuntu or Debian server with root or sudo privileges.
2. A FQDN (Fully Qualified Domain Name) for your mail server, e.g. mail.example.com.
3. DNS records pointing mail.example.com to your server’s IP.
4. Open ports on your firewall:

• 25 (SMTP inbound), 587 (SMTP submission), 993 (IMAP), 995 or 110 (POP3, optional), 80/443 for Let’s Encrypt if using web-based ACME.

1. SSL certificate (Let’s Encrypt or other). This guide shows optional Let’s Encrypt steps in Section 8.

Make sure your server’s hostname is set correctly:

sudo hostnamectl set-hostname mail.example.com

3. Install Required Packages

Update your system and install essential mail-related packages:

sudo apt-get update
sudo apt-get install postfix postfix-mysql dovecot-core dovecot-imapd dovecot-pop3d \
  dovecot-mysql mysql-server libsasl2-2 libsasl2-modules pwgen \
  apache2 php php-mysql # (for Roundcube via Apache/PHP)

Note: If you prefer Nginx or another web server, install that instead of Apache.

During Postfix installation, it may ask “General type of mail configuration”—choose “Internet Site” or something minimal, since we’ll override it with our custom configs.

4. Configure MySQL for Virtual Domains & Users

4.1 Create Database & User

If not done already:

sudo mysql -u root -p

Then:

CREATE DATABASE mailserver;
CREATE USER 'mail_admin'@'localhost' IDENTIFIED BY 'SomeStrongPassword';
GRANT ALL PRIVILEGES ON mailserver.* TO 'mail_admin'@'localhost';
FLUSH PRIVILEGES;
EXIT;

4.2 Create Tables

Inside mailserver, we usually have:

• A domain table for each domain (e.g. example.com).
• A mailbox table for each user/mailbox.
• An alias (or forwardings) table for any aliases.

For example:

USE mailserver;

CREATE TABLE domain (
  domain VARCHAR(50) NOT NULL,
  PRIMARY KEY (domain)
);

CREATE TABLE mailbox (
  username VARCHAR(100) NOT NULL,
  password VARCHAR(255) NOT NULL,
  name VARCHAR(100) NOT NULL,
  maildir VARCHAR(255) NOT NULL,
  quota INT(10) NOT NULL DEFAULT 0,
  local_part VARCHAR(64) NOT NULL,
  domain VARCHAR(50) NOT NULL,
  created DATETIME NOT NULL,
  modified DATETIME NOT NULL,
  active TINYINT(1) NOT NULL DEFAULT '1',
  PRIMARY KEY (username)
);

CREATE TABLE alias (
  address VARCHAR(100) NOT NULL,
  goto TEXT NOT NULL,
  domain VARCHAR(50) NOT NULL,
  created DATETIME NOT NULL,
  modified DATETIME NOT NULL,
  active TINYINT(1) NOT NULL DEFAULT '1',
  PRIMARY KEY (address)
);

Adjust sizes/columns as needed.

4.3 Insert Your Domain

INSERT INTO domain (domain) VALUES ('example.com');

4.4 MySQL Queries for Postfix & Dovecot

We’ll reference these .cf files in Postfix’s config. They typically contain lines like:

/etc/postfix/mysql-virtual-mailbox-domains.cf:

user = mail_admin
password = SomeStrongPassword
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM domain WHERE domain='%s'

/etc/postfix/mysql-virtual-mailbox-maps.cf:

user = mail_admin
password = SomeStrongPassword
hosts = 127.0.0.1
dbname = mailserver
query = SELECT maildir FROM mailbox WHERE username='%s' AND active=1

/etc/postfix/mysql-virtual-alias-maps.cf:

user = mail_admin
password = SomeStrongPassword
hosts = 127.0.0.1
dbname = mailserver
query = SELECT goto FROM alias WHERE address='%s' AND active=1

Make sure each file is owned by root:root and has 0600 or similarly restrictive permissions.

5. Configure Postfix

5.1 Main Configuration (/etc/postfix/main.cf)

Key directives (some lines might already exist):

myhostname = mail.example.com
mydomain = example.com
myorigin = /etc/mailname
mydestination = $myhostname, localhost.$mydomain, localhost
relayhost =

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
append_dot_mydomain = no
biff = no
inet_interfaces = all
inet_protocols = all

# Virtual domains
virtual_mailbox_base = /var/vmail
virtual_uid_maps = static:5000
virtual_gid_maps = static:5000
virtual_transport = lmtp:unix:private/dovecot-lmtp

virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf

# TLS
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_security_level = may
smtpd_tls_auth_only = yes
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_mandatory_protocols = !SSLv2,!SSLv3,!TLSv1,!TLSv1.1

# Authentication
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
smtpd_recipient_restrictions = permit_mynetworks,permit_sasl_authenticated,reject_unauth_destination

Adjust to match your paths and Let’s Encrypt cert. If you’re not using Let’s Encrypt, point to your existing SSL cert/key.

5.2 Master Configuration (/etc/postfix/master.cf)

Make sure submission (port 587) is enabled:

submission inet n       -       y       -       -       smtpd
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_security_level=encrypt
  -o smtpd_tls_wrappermode=no
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject_unauth_destination
  -o smtpd_sasl_security_options=noanonymous
  -o smtpd_sasl_local_domain=$myhostname
  -o smtpd_tls_auth_only=yes

Make sure indentation is correct: the line submission inet ... has no leading spaces, each -o line is indented by at least one space.

Then restart Postfix:

sudo systemctl restart postfix

6. Configure Dovecot

6.1 Dovecot main config: /etc/dovecot/dovecot.conf

Likely references conf.d. Ensure protocols are correct:

protocols = imap pop3 lmtp

6.2 10-auth.conf

disable_plaintext_auth = yes
auth_mechanisms = plain login

!include auth-sql.conf.ext

6.3 10-master.conf

Check the service auth & service lmtp sections:

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
    user = postfix
    group = postfix
  }
  user = dovecot
}

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0666
    user = postfix
    group = postfix
  }
}

This ensures Postfix can communicate with Dovecot for both AUTH and LMTP.

6.4 auth-sql.conf.ext

passdb {
  driver = sql
  args = /etc/dovecot/dovecot-sql.conf.ext
}

userdb {
  driver = static
  args = uid=vmail gid=vmail home=/var/vmail/%d/%n
}

6.5 dovecot-sql.conf.ext

driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=mail_admin password=SomeStrongPassword
default_pass_scheme = SHA512-CRYPT

password_query = SELECT username as user, password FROM mailbox WHERE username = '%u' AND active=1

Adjust pass scheme as needed if your mailbox table uses a different hash.

Restart Dovecot:

sudo systemctl restart dovecot

7. Configure Roundcube

7.1 Install Roundcube

On Ubuntu/Debian:

sudo apt-get install roundcube roundcube-core roundcube-mysql roundcube-plugins

It may prompt for initial DB config. Let it create a DB for Roundcube (separate from mailserver).

7.2 Adjust /etc/roundcube/config.inc.php

Example settings:

$config['db_dsnw'] = 'mysql://roundcube:RoundcubeDBPass@localhost/roundcube';

$config['default_host'] = 'ssl://mail.example.com:993';
$config['smtp_server']  = 'mail.example.com';
$config['smtp_port']    = 587;
$config['smtp_user']    = '%u';
$config['smtp_pass']    = '%p';
$config['smtp_secure']  = 'tls';

$config['product_name'] = 'My Webmail';
$config['des_key'] = 'ReplaceWith24RandomChars';

$config['plugins'] = array();

7.3 Make Sure the Web Server is Enabled

For Apache, ensure /etc/apache2/conf-enabled/roundcube.conf or similar is in place. Then:

sudo systemctl restart apache2

Visit http://your_server/roundcube or https://your_server/roundcube.

8. SSL Certificates with Let’s Encrypt (Optional)

If you used Let’s Encrypt:

sudo apt-get install certbot
sudo certbot certonly --apache -d mail.example.com

or with Nginx: --nginx. Then set smtpd_tls_cert_file and smtpd_tls_key_file in Postfix (and ssl_cert/ssl_key in Dovecot) to point to the Let’s Encrypt cert paths.

9. Testing the Setup

1. Check Postfix is listening on ports 25 & 587:

sudo ss -plnt | grep ':25\|:587'

1. Check Dovecot on ports 143/993:

sudo ss -plnt | grep ':993\|:143'

1. IMAP test:

openssl s_client -connect mail.example.com:993
a login bob@example.com yourpassword
a logout

1. SMTP test with swaks:

swaks --to test@otherdomain.com --from bob@example.com --server mail.example.com:587 \
      --auth LOGIN --auth-user bob@example.com --auth-password 'yourpass' --tls

1. If it says Authentication successful and the mail is queued, you’re good.
2. Roundcube test:

• Go to https://mail.example.com/roundcube.
• Log in with bob@example.com + your pass.
• Send a test email. Check logs in /var/log/mail.log and /var/log/roundcube/errors.log.

10. Managing Users

10.1 Creating a New Mail User

• Generate a hashed password:

doveadm pw -s SHA512-CRYPT

• Insert into MySQL:

USE mailserver;
INSERT INTO mailbox (username, password, name, maildir, quota, local_part, domain, created, modified, active)
VALUES (
  'alice@example.com',
  '{SHA512-CRYPT}$6$...',
  'Alice Smith',
  'example.com/alice/',
  0,
  'alice',
  'example.com',
  NOW(),
  NOW(),
  1
);

• Domain must exist in the domain table.
• That’s it—Postfix & Dovecot will pick it up immediately.

10.2 Aliases

If you want alias@domain.com to deliver into bob@domain.com:

INSERT INTO alias (address, goto, domain, created, modified, active)
VALUES (
  'alias@domain.com',
  'bob@domain.com',
  'domain.com',
  NOW(),
  NOW(),
  1
);

11. Enabling Sent Folders & Basic Roundcube Customization

11.1 Sent Folder Setup

Roundcube ? Settings ? Preferences ? Composing Messages:

• Save sent messages ? “Sent” folder.
• Subscribe to “Sent” in Roundcube’s Settings ? Folders.

11.2 Auto-Creation of Folders in Dovecot

/etc/dovecot/conf.d/15-mailboxes.conf example:

namespace inbox {
  inbox = yesmailbox Drafts {
    special_use = \Draftsauto = create
  }
  mailbox Sent {
    special_use = \Sentauto = create
  }
  mailbox Junk {
    special_use = \Junkauto = create
  }
  mailbox Trash {
    special_use = \Trashauto = create
  }
}

11.3 Adding an Outbox (Optional)

You can create a normal IMAP folder called “Outbox.” However, Roundcube does not queue mail there by default. If you want true “send later” or “outbox” functionality, you need a plugin or custom behavior.

12. Troubleshooting Tips

• Check mail logs:
• /var/log/mail.log or /var/log/maillog for Postfix & Dovecot messages.
• /var/log/roundcube/errors.log for Roundcube logs.
• Verify Postfix is offering AUTH:

openssl s_client -starttls smtp -connect mail.example.com:587
EHLO test
# Must see: 250-AUTH PLAIN LOGIN

• Ensure Dovecot’s auth_mechanisms = plain login, plus the socket is at /var/spool/postfix/private/auth with correct permissions.
• MySQL queries: use postmap -q youruser@example.com mysql:/etc/postfix/mysql-...cf to confirm Postfix sees the right maildir or alias.
• SELinux/AppArmor: if in “enforcing” mode, may block the socket. Consider checking or disabling for testing.
• For Roundcube issues, enable $config['smtp_debug'] = true; to see exactly how Roundcube is talking to your SMTP server.

Conclusion

You now have a complete mail server with:

1. Postfix for SMTP inbound/outbound.
2. Dovecot for IMAP/POP3 and user authentication (using MySQL).
3. Roundcube providing a modern webmail interface.

You can manage new domains and users simply by inserting rows into the MySQL domain, mailbox, and alias tables. Roundcube’s settings let you control sent/drafts folders, and with Dovecot’s auto-creation, users get a straightforward mail experience. Keep an eye on your SSL certificates, watch your logs for errors, and enjoy your fully functional mail system!

Login to Dev Server

eg: Putty to 192.168.1.109

Install Docker:

# Install prerequisites sudo apt install -y apt-transport-https ca-certificates curl software-properties-common # Add Docker's official GPG key curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg # Add Docker's repository to APT sourcesecho"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null # Update package lists again sudo apt update

Install Docker Compose

sudo apt install docker-compose-plugin

Verify Installation

Check if the plugin is installed:

bash

Copy code
docker compose version

Use docker compose Command

Try running your Compose file again:

bash

Copy code
docker compose up

NGINX Example

1. Install NGINX:

bash

Copy code
sudo apt install nginx

Configure NGINX for n8n: Create a file /etc/nginx/sites-available/n8n with the following content:

nginx

Copy code
server { listen 80; server_name <your-public-domain>; location / { proxy_pass http://localhost:5678; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; client_max_body_size 50M; } }

Enable the site and reload NGINX:

bash

Copy code
sudo ln -s /etc/nginx/sites-available/n8n /etc/nginx/sites-enabled/ sudo nginx -t sudo systemctl reload nginx

If you get an 400 error this is the fix:

server {

  # Existing configuration...

  client_header_buffer_sTest the configuration for syntax errors:

bash

Copy code
sudo nginx -t

Reload NGINX to apply changes:

bash

Copy code
sudo systemctl reload nginx

ize 16k;

  large_client_header_buffers 4 32k;

}

# Install SeedDMS Guide

## Step 1: System Requirements

Ensure your server meets the following requirements:

- Ubuntu 20.04

- Apache Web Server

- PHP version 7.4 or above

- MySQL installed

- SSL certificates configured (Let's Encrypt or other)

## Step 2: Download SeedDMS

Download SeedDMS version 6.0.28 from the official website or use the following command:

```bash

wget https://sourceforge.net/projects/seeddms/files/seeddms-quickstart-6.0.28.tar.gz

```

## Step 3: Extract Files

Create the directory where you want to extract SeedDMS and extract the downloaded tarball:

```bash

mkdir /var/www/seeddms-6.0.28
sudo tar -xzvf seeddms-quickstart-6.0.28.tar.gz -C /var/www/seeddms-6.0.28 --strip-components=1

```

## Step 4: Configure Apache

Create a new Apache configuration file for SeedDMS. For example, `/etc/apache2/sites-available/seeddms.conf`:

```apache

 ServerAdmin webmaster@localhost
 DocumentRoot /var/www/seeddms-6.0.28/www
 ServerName dms.anita.flast.com.au
 ServerAlias dms.anita.flast.com.au

 
   Options Indexes FollowSymLinks
   AllowOverride All
   Require all granted
   DirectoryIndex install.php index.php
 

 ErrorLog ${APACHE_LOG_DIR}/seeddms_error.log
 CustomLog ${APACHE_LOG_DIR}/seeddms_access.log combined

```

Add the SSL configuration to your existing SSL configuration file or create a new one:

```apache


  ServerAdmin webmaster@localhost
  ServerName dms.anita.flast.com.au
  ServerAlias dms.anita.flast.com.au
  DocumentRoot /var/www/seeddms-6.0.28/www

  
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
    DirectoryIndex install.php index.php
  

  SSLEngine on
  SSLCertificateFile /etc/letsencrypt/live/anita.flast.com.au/fullchain.pem
  SSLCertificateKeyFile /etc/letsencrypt/live/anita.flast.com.au/privkey.pem
  Include /etc/letsencrypt/options-ssl-apache.conf

  ErrorLog ${APACHE_LOG_DIR}/seeddms_ssl_error.log
  CustomLog ${APACHE_LOG_DIR}/seeddms_ssl_access.log combined

```

## Step 5: Enable Apache Configuration

Enable the newly created configuration and restart Apache:

```bash
sudo a2ensite seeddms.conf
sudo systemctl restart apache2

```

## Step 6: Set Permissions

Ensure that the appropriate permissions are set for the SeedDMS directory:

```bash
sudo chown -R www-data:www-data /var/www/seeddms-6.0.28/
sudo chmod -R 755 /var/www/seeddms-6.0.28/

```

## Step 7: Install SeedDMS

Navigate to the installation script in your web browser:

```

https://dms.anita.flast.com.au/install/install.php

```

If prompted, create the `ENABLE_INSTALL_TOOL` file in the `conf` directory:

```bash

touch /var/www/seeddms-6.0.28/conf/ENABLE_INSTALL_TOOL

```

Follow the on-screen instructions to complete the installation.

## Step 8: Post Installation

After the installation is completed, remove the `ENABLE_INSTALL_TOOL` file to prevent unauthorized changes:

```bash

rm /var/www/seeddms-6.0.28/conf/ENABLE_INSTALL_TOOL

```

Log in to SeedDMS using the default credentials (`admin/admin`) and configure your system settings as needed.

## Troubleshooting

- **404 Not Found Error**: Ensure that the `DocumentRoot` and `` paths in the Apache configuration point to the correct directory where SeedDMS files are located.
- **ERR\_TOO\_MANY\_REDIRECTS**: Check the `.htaccess` file or Apache configuration for conflicting redirects. Make sure the `DirectoryIndex` is correctly specified.

- **Missing ****`create_tables-innodb.sql`**** Error**: Verify that the SQL file is present in the `/install` directory, and try running the command manually:

 ```bash

 mysql -u seeddms -p seeddms < /var/www/seeddms-6.0.28/install/create_tables-innodb.sql

 ```

Config File

   siteName = "Anita Document Management System"
   footNote = "Anita Document Management System"
   printDisclaimer = "true"
   language = "en_GB"
   theme = "bootstrap4"
   previewWidthList = "40"
   previewWidthDetail = "100"
   onePageMode="true"
  />
  
   strictFormCheck = "false"
   viewOnlineFileTypes = ".txt;.text;.html;.htm;.xml;.pdf;.gif;.png;.jpg;.jpeg;.mp4"
   enableConverting = "true"
   enableEmail = "true"
   enableUsersView = "true"
   enableFullSearch = "true"
   enableClipboard = "true"
   enableFolderTree = "true"
   expandFolderTree = "1"
   enableLanguageSelector = "true"
   stopWordsFile = ""
   sortUsersInList = ""
   enableDropUpload = "true"
   enableRecursiveCount = "true"
   maxRecursiveCount = "0"
   enableThemeSelector = "true"
   fullSearchEngine = "sqlitefts"
   sortFoldersDefault = "u"
   defaultDocPosition = "end"
   defaultFolderPosition = "end"
  />
  
   enableCalendar = "true"
   calendarDefaultView = "y"
   firstDayOfWeek = "0"
  />
  
   enableWebdavReplaceDoc="true"
  />
 
 
  
   rootDir = "/var/www/html/dms/"
   httpRoot = "/dms/"
   contentDir = "/var/www/html/dms/data/"
   stagingDir = "/var/www/html/dms/data/staging/"
   luceneDir = "/var/www/html/dms/data/lucene/"
   logFileEnable = "true"
   logFileRotation = "d"
   enableLargeFileUpload = "true"
   partitionSize = "2000000"
   dropFolderDir = "/var/www/html/dms/data/drop/"
   cacheDir = "/var/www/html/dms/data/cache/"
   backupDir = "/var/www/html/dms/data/backup"
   debugLevel="3"
  />
  
   enableGuestLogin = "false"
   enablePasswordForgotten = "false"
   restricted = "true"
   enableUserImage = "false"
   disableSelfEdit = "false"
   disableChangePassword = "false"
   passwordStrength = "0"
   passwordStrengthAlgorithm = "simple"
   passwordExpiration = "0"
   passwordHistory = "0"
   loginFailure = "0"
   autoLoginUser = "0"
   quota = "0"
   undelUserIds = ""
   encryptionKey = "XXXXXXXXXXXXXXXXXX"
   cookieLifetime = "0">
   
    
     enable = "false"
     type = "ldap"
     host = "ldaps://ldap.host.com"
     port = "389"
     baseDN = ""
     bindDN = ""
     bindPw = ""
     filter = ""
     groupField = ""
    />
    
     enable = "false"
     type = "AD"
     host = "ldap.example.com"
     port = "389"
     baseDN = ""
     accountDomainName = "example.com"
     bindDN = ""
     bindPw = ""
     filter = ""
     groupField = ""
    />
   
  
  
   ADOdbPath="/var/www/html/dms/pear" 
   dbDriver="mysql" 
   dbHostname="localhost" 
   dbDatabase="XXXXXXX" 
   dbUser="XXXXXXX" 
   dbPass="XXXXXXX" 
  />
  
   smtpServer = "localhost"
   smtpPort = "25"
   smtpSendFrom = "anita@localhost"
   smtpUser = ""
   smtpPassword = ""
  />
 
 
  
   siteDefaultPage = ""
   rootFolderID = "1"
   showMissingTranslations = "false"
  />
  
   guestID = "2"
   adminIP = ""
  />
  
   enableAdminRevApp = "false"
   versioningFileName = "versioning_info.txt"
   workflowMode = "traditional"
   enableVersionDeletion = "true"
   enableVersionModification = "true"
   enableDuplicateDocNames = "true"
   enableDuplicateSubFolderNames = "true"
   enableOwnerRevApp = "false"
   enableSelfRevApp = "false"
   presetExpirationDate = ""
   overrideMimeType = "false"
  />
  
   coreDir = ""
   luceneClassDir = ""
   contentOffsetDir = "1048576"
   maxDirID = "0"
   updateNotifyTime = "86400"
   extraPath = ""
   maxExecutionTime = "30"
   cmdTimeout = "10"
  />
  
   enableNotificationAppRev = "true"
   enableOwnerNotification = "false"
   enableNotificationWorkflow = "false"
   />
  
   pdftotext -nopgbrk %s - | sed -e 's/ [a-zA-Z0-9.]\{1\} / /g' -e 's/[0-9.]//g'
   catdoc %s
   ssconvert -T Gnumeric_stf:stf_csv -S %s fd://1
   id3 -l -R %s | egrep '(Title|Artist|Album)' | sed 's/^[^:]*: //g'
   id3 -l -R %s | egrep '(Title|Artist|Album)' | sed 's/^[^:]*: //g'
   cat %s
   html2text %s
   docx2txt %s -
  
  
   unoconv -d document -e PageRange=1 -f pdf --stdout -v '%f'|gs -dBATCH -dNOPAUSE -sDEVICE=pngalpha -dPDFFitPage -r72x72 -sOutputFile=- -dFirstPage=1 -dLastPage=1 -q - | convert -resize %wx png:- '%o'
   convert -resize %wx '%f' '%o'
   convert -resize %wx '%f' '%o'
   convert -resize %wx '%f' '%o'
   a2ps -1 -a1 -R -B -o - '%f' | gs -dBATCH -dNOPAUSE -sDEVICE=pngalpha -dFirstPage=1 -dLastPage=1 -dPDFFitPage -r72x72 -sOutputFile=- -q - | convert -resize %wx png:- '%o'
  
 <server
 rootDir="/var/www/seeddms-6.0.28/"
 httpRoot="/"
 contentDir="data/"
 stagingDir="data/staging/"
 luceneDir="data/lucene/"
 logFileEnable="true"
 logFileRotation="d"
 enableLargeFileUpload="true"
 partitionSize="2000000"
 dropFolderDir="data/drop/"
 cacheDir="data/cache/"
 backupDir="data/backup"
 ADOdbPath="/var/www/seeddms-6.0.28/pear"
/>

database
 type="mysql"
 host="localhost"
 database="seeddms"
 user="seeddms"
 password="your_password_here"
/>
  

Proxmox VM Network

Subnet = IP with .0/24 not 255.255.255.0/24

Gatway is .254 always

Network with Bridge

edit /etc/netplan

copy and back up existing file.

create a new file with nano

insert :

network:
  version: 2
  ethernets:
    eno3:
      dhcp4: no # Disable DHCP since you have static IPs
      addresses:
        - 142.44.212.192/24 # Your main static IP
        - 54.39.92.189/32  # Additional IP 1
        - 54.39.92.191/32  # Additional IP 2
        - 54.39.118.47/32  # Additional IP 3
        - 54.39.118.48/32  # Additional IP 4
        - 2607:5300:203:25c0::1/64
      routes:
        - to: 0.0.0.0/0  # Default IPv4 route
         via: 142.44.212.254
        - to: ::/0  # Default IPv6 route
         via: 2607:5300:203:25ff:ff:ff:ff:ff # Or your actual IPv6 gateway
      nameservers:
        addresses: [8.8.8.8, 1.1.1.1, 2001:41d0:3:163::1]
      accept-ra: false
      match:
        macaddress: 0c:c4:7a:c8:76:ae

Ping to test

ping -I 54.39.92.191 8.8.8.8 

For example you have been assigned :

54.39.92.191

and

54.39.92.189

Here is ip a readout:

root@contractout:/var/lib/vz/template/iso# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group defaul                                                                                                                                                                                                                                             t qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state                                                                                                                                                                                                                                              UP group default qlen 1000
    link/ether 0c:c4:7a:c8:76:ae brd ff:ff:ff:ff:ff:ff
    altname enp3s0f0
3: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen                                                                                                                                                                                                                                              1000
    link/ether 0c:c4:7a:c8:76:af brd ff:ff:ff:ff:ff:ff
    altname enp3s0f1
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP grou                                                                                                                                                                                                                                             p default qlen 1000
    link/ether 0c:c4:7a:c8:76:ae brd ff:ff:ff:ff:ff:ff
    inet 142.44.212.192/24 scope global vmbr0
       valid_lft forever preferred_lft forever
5: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast                                                                                                                                                                                                                                              master fwbr100i0 state UNKNOWN group default qlen 1000
    link/ether 2e:71:17:63:af:19 brd ff:ff:ff:ff:ff:ff
6: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP                                                                                                                                                                                                                                              group default qlen 1000
    link/ether 5a:40:8e:1b:8a:e7 brd ff:ff:ff:ff:ff:ff
7: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue                                                                                                                                                                                                                                              master vmbr0 state UP group default qlen 1000
    link/ether be:6d:0e:18:91:8d brd ff:ff:ff:ff:ff:ff
8: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue                                                                                                                                                                                                                                              master fwbr100i0 state UP group default qlen 1000
    link/ether 5a:40:8e:1b:8a:e7 brd ff:ff:ff:ff:ff:ff
root@contractout:/var/lib/vz/template/iso# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether 0c:c4:7a:c8:76:ae brd ff:ff:ff:ff:ff:ff
    altname enp3s0f0
3: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 0c:c4:7a:c8:76:af brd ff:ff:ff:ff:ff:ff
    altname enp3s0f1
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 0c:c4:7a:c8:76:ae brd ff:ff:ff:ff:ff:ff
    inet 142.44.212.192/24 scope global vmbr0
       valid_lft forever preferred_lft forever
5: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr100i0 state UNKNOWN group default qlen 1000
    link/ether 2e:71:17:63:af:19 brd ff:ff:ff:ff:ff:ff
6: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5a:40:8e:1b:8a:e7 brd ff:ff:ff:ff:ff:ff
7: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether be:6d:0e:18:91:8d brd ff:ff:ff:ff:ff:ff
8: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
    link/ether 5a:40:8e:1b:8a:e7 brd ff:ff:ff:ff:ff:ff

Next you need to :

sudo nano /etc/network/interfaces

1. Add the new IP addresses: Add the new IP addresses to the vmbr0 interface. Your configuration should look something like this:

auto lo
iface lo inet loopback

iface eno3 inet manual

auto vmbr0
iface vmbr0 inet static
    address 142.44.212.192/24
    gateway 142.44.212.254
    bridge-ports eno3
    bridge-stp off
    bridge-fd 0
    hwaddress 0C:C4:7A:C8:76:AE

iface vmbr0 inet6 static
    address 2607:5300:203:25c0::1/128
    gateway 2607:5300:203:25ff:ff:ff:ff:ff

# Add the new IP addresses here
up ip addr add 54.39.92.191/24 dev vmbr0
up ip addr add 54.39.92.189/24 dev vmbr0

ave and exit: Save the file and exit the text editor (in nano, you can do this by pressing Ctrl+X, then Y, and Enter).

Restart the networking service: Restart the networking service to apply the changes:

sudo systemctl restart networking

Verify the new IP addresses: Check if the new IP addresses have been added successfully:

ip addr show vmbr0

This should add the two new IP addresses to your vmbr0 interface.

Proxmox

SSH in 

su - = root

apt-get update = first time it may error 

root@contractout:~# rm /etc/apt/sources.list.d/pve-enterprise.list

root@contractout:~# nano /etc/apt/sources.list

root@contractout:~# apt-get update

Now let's give the user access:

root@contractout:~# usermod -aG sudo danny

root@contractout:~# sudo whoami root

root@contractout:~# sudo systemctl restart pveproxy.service

root@contractout:~# sudo systemctl restart pvedaemon.service

root@contractout:~# pveum user add danny@pam

root@contractout:~# pveum acl modify / --roles PVEAdmin --users danny@pam

Upload Images Directly 

root@contractout:~# cd /var/lib/vz/template/iso/

root@contractout:/var/lib/vz/template/iso# ls

root@contractout:/var/lib/vz/template/iso# wget https://releases.ubuntu.com/20.04.6/ubuntu-20.04.6-live-server-amd64.iso

Here are the steps to install Elasticsearch on Ubuntu:

1. Update your system: Use the following command to update your system:

sudo apt update

1. Import the Elasticsearch public GPG key into APT: You can use cURL for this12. Here is the command:

curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

1. Add the Elasticsearch source list to the sources.list.d directory12. Use this command:

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

1. Update your package lists so APT will read the new Elastic source12. Here is the command:

sudo apt update

1. Install Elasticsearch using this command12:

sudo apt install elasticsearch

1. Reload the daemon services and enable Elasticsearch to start on boot12.
2. Start the Elasticsearch service and check its status12.
3. Edit the elasticsearch.yml file and configure the host IP, port, and discovery settings2.
4. Restart the Elasticsearch service and check its status again12.
5. Go to the browser and enter the IP and port of the host to see the Elasticsearch configuration2.

To clone your private repository to another server, you'll need to use a method that allows authentication. Here are the two main approaches:

1. SSH (Recommended):

This is the most secure and convenient way to clone a private repository.

Prerequisites:

• SSH Key Pair on the New Server: Generate an SSH key pair on the server where you want to clone the repository (if it doesn't have one already). You can use ssh-keygen for this.
• Public Key Added to GitHub: Make sure the public key from the new server is added to the list of "SSH and GPG keys" in your GitHub account settings.

Steps:

1. Get the SSH URL:

• Go to your GitHub repository.

Click the "Code" button and select the "SSH" tab to copy the SSH URL. It should look like this:git@github.com:cr0atz/flast-ai-v5.1.git

1. Clone the Repository:

• On the new server, open a terminal and run:Bash

git clone git@github.com:cr0atz/flast-ai-v5.1.git

• content_copy
• This will clone the repository using SSH authentication.

2. Personal Access Token (PAT):

This method is an alternative if you don't want to use SSH keys.

Prerequisites:

• Personal Access Token (PAT): Generate a PAT in your GitHub account settings with the "repo" scope.

Steps:

• Clone the Repository:On the new server, run the following command, replacing <your_pat> with your actual PAT:Bash

git clone https://<your_pat>@github.com/cr0atz/flast-ai-v5.1.git

• content_copy
• This will prompt you for your GitHub username (which you can enter). Git will use the PAT you've included in the URL for authentication.

Important Considerations:

• HTTPS vs. SSH: The SSH method uses the git@github.com URL, while the PAT method uses the https://github.com URL.
• Security: SSH keys are generally more secure than embedding your PAT directly in the URL.
• 2FA: If you have two-factor authentication (2FA) enabled on GitHub, you'll be prompted for a 2FA code during the cloning process if you use the PAT method.

Additional Tips:

• Virtual Environments: If your project uses a virtual environment, remember to create and activate a new virtual environment on the new server before installing dependencies.
• Troubleshooting: If you encounter permission errors, ensure that the user running the git clone command has the necessary permissions to create the repository directory and files on the new server.

Let me know if you have any other questions!

First of all if running the YT - transcriber we need to make sure the service is running within the venv environment.

[Unit]
Description= flastai engine client
After=network.target

[Service]
User=flastai
WorkingDirectory=/var/www/html/BlogEngine_v5.0/

# ExecStart=/var/www/html/BlogEngine_v5.0/venv/bin/streamlit run home.py --server.port 30000
ExecStart=/bin/bash -c 'source venv/bin/activate && /var/www/html/BlogEngine_v5.0/venv/bin/streamlit run home.py --server.port 30000'

Restart=always

[Install]
WantedBy=multi-user.target

New Code (inside home.py) :

class Document:
    def __init__(self):
        self.text = ""


    def set_text(self, text):
        self.text = text


def transcribing_function(video_id):
    st.warning("Transcribing video...")


    
    # Define the path to the Whisper model files and audio folder
    whisper_model_path = f"../{folder_root}/"
    audio_path = f"{folder_audio}/{video_id}.mp3"
    
    # Execute yt-dlp command to download and convert video audio to mp3
    result = subprocess.run(
        [yt_dlp_path, '-f', 'bestaudio', '-o', audio_path, f"https://www.youtube.com/watch?v={video_id}"],
        capture_output=True,
        text=True
    )
    
    # Check for errors in the subprocess execution
    if result.returncode != 0:
        st.error("Failed to download and convert video.")
        st.error(result.stderr)
        return


    # Load Whisper model
    model = whisper.load_model("base.en", download_root=whisper_model_path)


    # Transcribe the audio file
    try:
        transcription_result = model.transcribe(audio_path)
        text = transcription_result['text'].strip()
    except Exception as e:
        st.error(f"Error transcribing video: {str(e)}")
        return


    # Split the text into parts
    max_chars = 28000
    parts = [text[i:i + max_chars] for i in range(0, len(text), max_chars)]
    docs = []
    for part in parts:
        doc = Document()
        doc.set_text(part)
        docs.append(doc)


    # Save each part to a separate file
    for i, doc in enumerate(docs):
        filename = os.path.join(folder_files, f"transcribed-{video_id}_{i}.txt")
        with open(filename, "w", encoding="utf-8") as f:
            f.write(doc.text)
        st.success(f"Transcribed chunk_{i} saved.")


    # Assuming right_column is defined correctly elsewhere
    with right_column:
        # Get and display transcribed files
        transcribed_files = [file for file in os.listdir(folder_files) if file.endswith(".txt") and video_id in file]
        if not transcribed_files:
            st.error("No transcribed files found.")
            return


        st.success("Video transcribed and split into parts successfully.")
        num_columns = min(len(transcribed_files), 3)
        columns = st.columns(num_columns, gap="small")


        for i, column in enumerate(columns):
            for j in range(i, len(transcribed_files), num_columns):
                file_path = os.path.join(folder_files, transcribed_files[j])
                with open(file_path, "rb") as f:
                    contents = f.read()
                    encoded = base64.b64encode(contents).decode()
                    href = f'<a href="data:application/octet-stream;base64,{encoded}" download="{transcribed_files[j]}" target="_blank">{transcribed_files[j]}</a>'
                    column.markdown(href, unsafe_allow_html=True)


        with st.expander("Transcribed files (Click to hide/unhide text)"):
            for filename in transcribed_files:
                with open(os.path.join(folder_files, filename), "r") as file:
                    file_contents = file.read()
                    st.success(f"File name: {filename}\n\n{file_contents}")


    return docs

Configuration - Requirements.

Activate the virtual environment:

Bash

 source /var/www/html/BlogEngine_v5.0/venv/bin/activate

Uninstall Whisper:Bash

pip uninstall whisper

Increase memory

sudo mount -o remount,size=6G /tmp

Reinstall Whisper: Make sure you're installing the latest version.

pip install -U openai-whisper --no-cache-dir

Tools :

• Cursor.sh
• GPT-4

This class you will learn how to:

• Set-Up a mini-conda environment
• deploy a Custom AI Instance

Set-Up MiniConda

You should do this as the user/domain folder which will be the host.

mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh

After installing, initialize your newly-installed Miniconda. The following commands initialize for bash and zsh shells:

~/miniconda3/bin/conda init bash
~/miniconda3/bin/conda init zsh

Restart the shell

exec bash

Source the updated .bash configuration

source ~/.bashrc

Create your conda environment

conda create --name aiinstancename python=3.11

Activate your conda enviroment

conda activate aiinstancename

Install dependancies

pip install streamlit
pip install openai

Set up the System Service

This you need to do as root

su -

Create the service eg: aiinstance.service

[Unit]
Description=v1 client
After=network.target


[Service]
User=flastcom
WorkingDirectory=/home/xxxcom/xxxServers/aiinstance
ExecStart=/home/xxxcom/miniconda3/envs/aiinstance/bin/python -m streamlit run /home/xxxcom/xxxServers/aiinstance/app_anyscale_streamlit.workswithprompt.py --server.port 35003
Restart=always


[Install]
WantedBy=multi-user.target

Enable - Start - Status

systemctl enable aiinstance.service
systemctl start aiinstance.service
systemctl status aiinstance.service

You should see your aiinstance running like this

[root@server:system]$ systemctl status v1-lmw.client.service
? v1-lmw.client.service - v1 client
   Loaded: loaded (/etc/systemd/system/v1-lmw.client.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2024-02-08 00:48:40 EST; 7s ago
 Main PID: 1822956 (python)
   CGroup: /system.slice/v1-lmw.client.service
           ??1822956 /home/xxxcom/miniconda3/envs/aiinstance/bin/python -m streamlit run /home/xxxcom/xxxServers/aiinstance/app_anyscale_streamlit.workswithprompt.py --server.port 35003


Feb 08 00:48:40 server.xxx.life systemd[1]: Started v1 client.
Feb 08 00:48:42 server.xxx.life python[1822956]: Collecting usage statistics. To deactivate, set browser.gatherUsageStats to False.
Feb 08 00:48:42 server.xxx.life python[1822956]:   You can now view your Streamlit app in your browser.
Feb 08 00:48:42 server.xxx.life python[1822956]:   Network URL: http://xxx.249.5.201:35003
Feb 08 00:48:42 server.xxx.life python[1822956]:   External URL: http://xxx.249.5.201:35003

Finally you need to use stunnel to push the http over https:

[File-AiInstance-domain]
accept = 35003  # this is the http port
connect = 35004 # this is the https port 
cert = /var/cpanel/ssl/apache_tls/xxx.com.au/combined  #these are the domains SSL certs locations 
key = /var/cpanel/ssl/apache_tls/xxx.com.au/combined  

If you did a rsync change the root password

sudo passwd root

cd to the working directory.

cd /var/www/html/BlogEngine_v5

edit the .env file ENDPOINT with the new domain

nano .env 

Run UVICORN and check for dependencies / errors

uvicorn engine:app --host 0.0.0.0 --port 8502 --reload

If you get an error about tensorflow - install it

pip install tensorflow

Run Streamlit in working directory to test.

streamlit run home.py --server.port 5173

Now let's re-configure the services

cd /etc/systemd/system

rename the service using cp command

cp name.client.service newname.client.service
cp name.server.service newname.server.service

Edit the new client service file: Change the Description and User

nano newname.client.service

Edit the new server service file: Change the Description and User

nano newname.server.service 

Edit the Streamlit.conf file in /etc/stunnel - Change the Name in the Top and the user in both SSL's

nano /etc/stunnel/streamlit.conf 

Create new User matching what you have in the Service files.

sudo adduser newuser

Now cd back to the working directory and let's change file permissions.

sudo chown -R newuser:newuser .

Change the permissions of engine.py, .env, home.py, images, prompts, .streamlit, and style to rwxrwxrwx (777):

sudo chmod 777 engine.py .env home.py 
sudo chmod -R 777 images prompts .streamlit style