Flask request extension and database connection pool

Flask request extension and database connection pool

1.1. Flask request extension

#!/usr/bin/env python
# -*- coding:utf-8 -*-
from flask import Flask, Request, render_template

app = Flask(__name__, template_folder='templates')
app.debug = True


@app.before_first_request
def before_first_request1():
    print('before_first_request1')


@app.before_first_request
def before_first_request2():
    print('before_first_request2')


@app.before_request
def before_request1():
    Request.nnn = 123
    print('before_request1')


@app.before_request
def before_request2():
    print('before_request2')


@app.after_request
def after_request1(response):
    print('before_request1', response)
    return response


@app.after_request
def after_request2(response):
    print('before_request2', response)
    return response


@app.errorhandler(404)
def page_not_found(error):
    return'This page does not exist', 404


@app.template_global()
def sb(a1, a2):
    return a1 + a2


@app.template_filter()
def db(a1, a2, a3):
    return a1 + a2 + a3


@app.route('/')
def hello_world():
    return render_template('hello.html')


if __name__ =='__main__':
    app.run()

(1) Do user login authentication based on before_request

@app.before_request
def process_request(*args,**kwargs):
    #Login page does not require verification
    if request.path =='/login':
        return None
    user = session.get('user_info')
    if user:
        return None
    return redirect('/login')

(2) The execution order of before_request and after_request

from flask import Flask,render_template,request,redirect,session,url_for

app = Flask(__name__)
app.debug = True
app.secret_key ='abcdef'



@app.before_request
def process_request1(*args,**kwargs):
    print('request1 comes in')

@app.before_request
def process_request2(*args,**kwargs):
    print('request2 comes in')

@app.after_request
def process_response1(response):
    print('response1 is gone')
    return response

@app.after_request
def process_response2(response):
    print('response2 is gone')
    return response

@app.route('/index',methods=['GET'])
def index():
    print('index function')
    return'hello'

if __name__ =='__main__':
    app.run()

operation result:

 As can be seen:

  • before_request is the order of execution from top to bottom (first 1 then 2)
  • after_response is the execution order from bottom to top (first 2 then 1)

(3) After the request is intercepted, all responses are executed

Add a return in process_request

@app.before_request
def process_request1(*args,**kwargs):
    print('request1 comes in')
    return "intercept"

Re-run results are as follows:

 (4) Custom error message

When visiting a non-existent url, you can customize the error message page by yourself

@app.errorhandler(404)
def error_404(arg):
    return '404 error'

1.2. Database connection pool

DBUtils is a Python module used to implement database connection pooling.

installation

Go to the official website to download https://pypi.org/project/DBUtils/1.2/, and then install:

There are two ways to connect pool

(1) Mode One

Create a connection for each thread. Even if the thread calls the close method, it will not be closed, but will put the connection back into the connection pool for its own thread to use again. When the thread terminates, the connection is automatically closed.

POOL = PersistentDB(
    creator=pymysql, # Use the module linked to the database
    maxusage=None, # The maximum number of times a link can be reused, None means unlimited
    setsession=[], # List of commands executed before starting the session. Such as: ["set datestyle to ...", "set time zone ..."]
    ping=0,
    # Ping the MySQL server to check whether the service is available. # Such as: 0 = None = never, 1 = default = whenever it is requested, 2 = when a cursor is created, 4 = when a query is executed, 7 = always
    closeable=False,
    # If it is False, conn.close() is actually ignored for the next use, and the link will be automatically closed when the thread is closed. If it is True, conn.close() will close the link, then an error will be reported when pool.connection is called again, because the connection has been really closed (pool.steady_connection() can get a new link)
    threadlocal=None, # This thread has an exclusive value object, used to save the link object, if the link object is reset
    host='127.0.0.1',
    port=3306,
    user='root',
    password='123',
    database='pooldb',
    charset='utf8'
)

def func():
    conn = POOL.connection(shareable=False)
    cursor = conn.cursor()
    cursor.execute('select * from tb1')
    result = cursor.fetchall()
    cursor.close()
    conn.close()

func()

(2) Mode two, the recommended way

Create multiple connections, when multiple threads come, get them,

import time
import pymysql
import threading
from DBUtils.PooledDB import PooledDB, SharedDBConnection
POOL = PooledDB(
    creator=pymysql, # Use the module linked to the database
    maxconnections=6, # The maximum number of connections allowed in the connection pool, 0 and None means no limit on the number of connections
    mincached=2, # When initializing, at least free links created in the link pool, 0 means not to create
    maxcached=5, # The most idle links in the link pool, 0 and None are not limited
    maxshared=3, # The maximum number of shared links in the link pool, 0 and None means all shared. PS: Useless, because the threadsafety of modules such as pymysql and MySQLdb is 1, no matter how many values ​​are set, _maxcached is always 0, so it is always shared by all links.
    blocking=True, # If there is no available connection in the connection pool, whether to block waiting. True, wait; False, do not wait and report an error
    maxusage=None, # The maximum number of times a link can be reused, None means unlimited
    setsession=[], # List of commands executed before starting the session. Such as: ["set datestyle to ...", "set time zone ..."]
    ping=0,
    # Ping the MySQL server to check whether the service is available. # Such as: 0 = None = never, 1 = default = whenever it is requested, 2 = when a cursor is created, 4 = when a query is executed, 7 = always
    host='127.0.0.1',
    port=3306,
    user='root',
    password='123',
    database='pooldb',
    charset='utf8'
)


def func():
    # Check whether the number of currently running connections is less than the maximum number of connections, if not less than: wait or report raise TooManyConnections exception
    # Otherwise
    # The priority is to get the link SteadyDBConnection from the link created during initialization.
    # Then encapsulate the SteadyDBConnection object into PooledDedicatedDBConnection and return.
    # If the link created at the beginning does not have a link, create a SteadyDBConnection object, then encapsulate it in PooledDedicatedDBConnection and return.
    # Once the link is closed, the connection is returned to the connection pool for subsequent threads to continue to use.
    conn = POOL.connection()

    # print(th,'The link was taken away', conn1._con)
    # print(th,'There are currently in the pool', pool._idle_cache,'\r\n')

    cursor = conn.cursor()
    cursor.execute('select * from tb1')
    result = cursor.fetchall()
    conn.close()


func()

If there is no connection pool, when using pymysql to connect to the database, there is no problem with single-threaded applications. However, if multi-threaded applications are involved, then locks are required. Once locked, the connection is bound to wait in line. When there are more requests, the performance Will be lowered.

import pymysql
import threading
from threading import RLock

LOCK = RLock()
CONN = pymysql.connect(host='127.0.0.1',
                       port=3306,
                       user='root',
                       password='123',
                       database='pooldb',
                       charset='utf8')


def task(arg):
    with LOCK:
        cursor = CONN.cursor()
        cursor.execute('select * from tb1')
        result = cursor.fetchall()
        cursor.close()

        print(result)


for i in range(10):
    t = threading.Thread(target=task, args=(i,))
    t.start()
Reference: https://cloud.tencent.com/developer/article/1139360 Flask request extension and database connection pool-Cloud + Community-Tencent Cloud