Compare commits

..

2 Commits

Author SHA1 Message Date
a00ed940d8
Update ER diagram 2020-01-10 23:29:36 +01:00
ee9558a1bb
Add plotting functionality 2020-01-10 23:29:23 +01:00
20 changed files with 224 additions and 94 deletions

View File

@ -11,8 +11,6 @@
*** TODO Improve second handout [0/2] [0%]
- [ ] Correct diagrams
- [ ] Add conceptual diagram
*** INACTIVE Update date in YAML automatically [0/1] [0%]
- [ ] Add to Makefile
*** DONE Type requirements handout [4/4] [100%]
CLOSED: [2019-09-27 Fri 14:54] SCHEDULED: <2019-09-27 Fri 23:55>
- [X] Problem description
@ -44,10 +42,15 @@ CLOSED: [2019-11-01 Fri 00:34]
- [X] Functional
- [X] Black box
- [X] Entity-Relationship
*** CANCELLED Update date in YAML automatically [0/1] [0%]
CLOSED: [2020-01-10 Fri 17:54]
- [ ] Add to Makefile
** Implementation
*** TODO Backend [3/4] [75%]
**** TODO Flask Application [2/3] [66%]
- [ ] Plots with pandas
*** DONE Backend [4/4] [100%]
CLOSED: [2020-01-10 Fri 23:15]
**** DONE Flask Application [3/3] [100%]
CLOSED: [2020-01-10 Fri 23:15]
- [X] Plots with pandas
- [X] Login for admin
- [X] Tables with pandas
**** DONE Database [3/3] [100%]
@ -64,10 +67,9 @@ CLOSED: [2020-01-08 Wed 03:18]
CLOSED: [2020-01-09 Thu 20:57]
- [X] Text search for glaciers
- [X] Form seach for a year
*** TODO Documentation [1/3] [33%]
- [X] Readme
*** TODO Documentation [1/2] [50%]
- [ ] Code
- [ ] Explanations (uid as Varchar, technologies used)
- [X] Readme
*** DONE Deployment [1/1] [100%]
CLOSED: [2020-01-10 Fri 12:14]
- [X] Installation instructions

View File

@ -8,7 +8,7 @@ consequences of climate change.
Our system allows you to visualize data with tables and plots, via our
intuitive Web UI.
![Table](./assets/screenshots/Table.png){width = 50%}
![Table](./assets/screenshots/Table.png)
Technologies used
-----------------

Binary file not shown.

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 17 KiB

View File

@ -26,13 +26,13 @@
\vfill
\noindent
\includegraphics[width=120pt, center]{/home/coolneng/Pictures/Logos/UGR.png}
\includegraphics[width=120pt, center]{/home/coolneng/Photos/Logos/UGR.png}
\vspace{\baselineskip}\noindent
\large
\begin{tabular}{lp{\textwidth}}
Autor: & \texttt{Amin Kasrou Aouam}\\
Fecha: & \texttt{18/10/2019}\\
Fecha: & \texttt{10/01/2020}\\
&\\[24pt]
\end{tabular}
}
@ -45,10 +45,4 @@
%------------------------------------- Workaround for CleanStyle -------------------
%% cleanthesis.sty *will* check the bibfile, even if `configurebiblatex=false` ...
%% So we need to set it appropriately using our metadata variable "cleanthesisbibfile"
\PassOptionsToPackage{
figuresep=colon,
configurelistings=true,
configurebiblatex=false,
bibfile=Assets/Citations
}{cleanthesis}
%------------------------------------- Workaround for CleanStyle -------------------

View File

@ -2,8 +2,8 @@
title: "IGDB: Base de datos internacional de glaciares"
subtitle: "Diseño y Desarrollo de Sistemas de Información"
author: [Amin Kasrou Aouam]
date: 18/10/2019
logo: /home/coolneng/Pictures/Logos/UGR.png
date: 10/01/2020
logo: /home/coolneng/Photos/Logos/UGR.png
lang: es-ES
toc: true
toc-own-page: true

View File

@ -14,6 +14,7 @@ iso3166 = "*"
flask-wtf = "*"
flask-login = "*"
flask-bootstrap = "*"
matplotlib = "*"
[requires]
python_version = "3.8"

77
code/Pipfile.lock generated
View File

@ -1,7 +1,7 @@
{
"_meta": {
"hash": {
"sha256": "b25fa5cfde97f34d3f4210b7c3e602df640d50105eb290daefa14c194b289936"
"sha256": "80f6c409aa1104974ff405c48d5527140030fb7e8d6bbf8aa21e884b3d151b5a"
},
"pipfile-spec": 6,
"requires": {
@ -23,6 +23,13 @@
],
"version": "==7.0"
},
"cycler": {
"hashes": [
"sha256:1d8a5ae1ff6c5cf9b93e8811e581232ad8920aeec647c37316ceac982b08cb2d",
"sha256:cd7b2d1018258d7247a71425e9f26463dfb444d411c39569972f4ce586b0c9d8"
],
"version": "==0.10.0"
},
"dominate": {
"hashes": [
"sha256:6e833aea505f0236a9fc692326bac575f8bd38ae0f3a1bdc73d20ca606ac75d5",
@ -90,6 +97,48 @@
],
"version": "==2.10.3"
},
"kiwisolver": {
"hashes": [
"sha256:05b5b061e09f60f56244adc885c4a7867da25ca387376b02c1efc29cc16bcd0f",
"sha256:210d8c39d01758d76c2b9a693567e1657ec661229bc32eac30761fa79b2474b0",
"sha256:26f4fbd6f5e1dabff70a9ba0d2c4bd30761086454aa30dddc5b52764ee4852b7",
"sha256:3b15d56a9cd40c52d7ab763ff0bc700edbb4e1a298dc43715ecccd605002cf11",
"sha256:3b2378ad387f49cbb328205bda569b9f87288d6bc1bf4cd683c34523a2341efe",
"sha256:400599c0fe58d21522cae0e8b22318e09d9729451b17ee61ba8e1e7c0346565c",
"sha256:47b8cb81a7d18dbaf4fed6a61c3cecdb5adec7b4ac292bddb0d016d57e8507d5",
"sha256:53eaed412477c836e1b9522c19858a8557d6e595077830146182225613b11a75",
"sha256:58e626e1f7dfbb620d08d457325a4cdac65d1809680009f46bf41eaf74ad0187",
"sha256:5a52e1b006bfa5be04fe4debbcdd2688432a9af4b207a3f429c74ad625022641",
"sha256:5c7ca4e449ac9f99b3b9d4693debb1d6d237d1542dd6a56b3305fe8a9620f883",
"sha256:682e54f0ce8f45981878756d7203fd01e188cc6c8b2c5e2cf03675390b4534d5",
"sha256:76275ee077772c8dde04fb6c5bc24b91af1bb3e7f4816fd1852f1495a64dad93",
"sha256:79bfb2f0bd7cbf9ea256612c9523367e5ec51d7cd616ae20ca2c90f575d839a2",
"sha256:7f4dd50874177d2bb060d74769210f3bce1af87a8c7cf5b37d032ebf94f0aca3",
"sha256:8944a16020c07b682df861207b7e0efcd2f46c7488619cb55f65882279119389",
"sha256:8aa7009437640beb2768bfd06da049bad0df85f47ff18426261acecd1cf00897",
"sha256:9105ce82dcc32c73eb53a04c869b6a4bc756b43e4385f76ea7943e827f529e4d",
"sha256:933df612c453928f1c6faa9236161a1d999a26cd40abf1dc5d7ebbc6dbfb8fca",
"sha256:939f36f21a8c571686eb491acfffa9c7f1ac345087281b412d63ea39ca14ec4a",
"sha256:9491578147849b93e70d7c1d23cb1229458f71fc79c51d52dce0809b2ca44eea",
"sha256:9733b7f64bd9f807832d673355f79703f81f0b3e52bfce420fc00d8cb28c6a6c",
"sha256:a02f6c3e229d0b7220bd74600e9351e18bc0c361b05f29adae0d10599ae0e326",
"sha256:a0c0a9f06872330d0dd31b45607197caab3c22777600e88031bfe66799e70bb0",
"sha256:aa716b9122307c50686356cfb47bfbc66541868078d0c801341df31dca1232a9",
"sha256:acc4df99308111585121db217681f1ce0eecb48d3a828a2f9bbf9773f4937e9e",
"sha256:b64916959e4ae0ac78af7c3e8cef4becee0c0e9694ad477b4c6b3a536de6a544",
"sha256:d22702cadb86b6fcba0e6b907d9f84a312db9cd6934ee728144ce3018e715ee1",
"sha256:d3fcf0819dc3fea58be1fd1ca390851bdb719a549850e708ed858503ff25d995",
"sha256:d52e3b1868a4e8fd18b5cb15055c76820df514e26aa84cc02f593d99fef6707f",
"sha256:db1a5d3cc4ae943d674718d6c47d2d82488ddd94b93b9e12d24aabdbfe48caee",
"sha256:e3a21a720791712ed721c7b95d433e036134de6f18c77dbe96119eaf7aa08004",
"sha256:e8bf074363ce2babeb4764d94f8e65efd22e6a7c74860a4f05a6947afc020ff2",
"sha256:f16814a4a96dc04bf1da7d53ee8d5b1d6decfc1a92a63349bb15d37b6a263dd9",
"sha256:f2b22153870ca5cf2ab9c940d7bc38e8e9089fa0f7e5856ea195e1cf4ff43d5a",
"sha256:f790f8b3dff3d53453de6a7b7ddd173d2e020fb160baff578d578065b108a05f",
"sha256:fe51b79da0062f8e9d49ed0182a626a7dc7a0cbca0328f612c6ee5e4711c81e4"
],
"version": "==1.1.0"
},
"markupsafe": {
"hashes": [
"sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
@ -123,6 +172,25 @@
],
"version": "==1.1.1"
},
"matplotlib": {
"hashes": [
"sha256:08ccc8922eb4792b91c652d3e6d46b1c99073f1284d1b6705155643e8046463a",
"sha256:161dcd807c0c3232f4dcd4a12a382d52004a498174cbfafd40646106c5bcdcc8",
"sha256:1f9e885bfa1b148d16f82a6672d043ecf11197f6c71ae222d0546db706e52eb2",
"sha256:2d6ab54015a7c0d727c33e36f85f5c5e4172059efdd067f7527f6e5d16ad01aa",
"sha256:5d2e408a2813abf664bd79431107543ecb449136912eb55bb312317edecf597e",
"sha256:61c8b740a008218eb604de518eb411c4953db0cb725dd0b32adf8a81771cab9e",
"sha256:80f10af8378fccc136da40ea6aa4a920767476cdfb3241acb93ef4f0465dbf57",
"sha256:819d4860315468b482f38f1afe45a5437f60f03eaede495d5ff89f2eeac89500",
"sha256:8cc0e44905c2c8fda5637cad6f311eb9517017515a034247ab93d0cf99f8bb7a",
"sha256:8e8e2c2fe3d873108735c6ee9884e6f36f467df4a143136209cff303b183bada",
"sha256:98c2ffeab8b79a4e3a0af5dd9939f92980eb6e3fec10f7f313df5f35a84dacab",
"sha256:d59bb0e82002ac49f4152963f8a1079e66794a4f454457fd2f0dcc7bf0797d30",
"sha256:ee59b7bb9eb75932fe3787e54e61c99b628155b0cedc907864f24723ba55b309"
],
"index": "pypi",
"version": "==3.1.2"
},
"numpy": {
"hashes": [
"sha256:1786a08236f2c92ae0e70423c45e1e62788ed33028f94ca99c4df03f5be6b3c6",
@ -182,6 +250,13 @@
"index": "pypi",
"version": "==0.9.3"
},
"pyparsing": {
"hashes": [
"sha256:4c830582a84fb022400b85429791bc551f1f4871c33f23e44f353119e92f969f",
"sha256:c342dccb5250c08d45fd6f8b4a559613ca603b57498511740e65cd11a2e7dcec"
],
"version": "==2.4.6"
},
"python-dateutil": {
"hashes": [
"sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c",

View File

@ -10,7 +10,7 @@ class LoginForm(FlaskForm):
submit = SubmitField("Sign In")
class YearForm(FlaskForm):
class AnnualForm(FlaskForm):
year_list = [
("2011", 2011),
("2012", 2012),
@ -26,22 +26,6 @@ class YearForm(FlaskForm):
submit = SubmitField("Search")
class IntervalForm(FlaskForm):
year_list = [
("2011", 2011),
("2012", 2012),
("2013", 2013),
("2014", 2014),
("2015", 2015),
("2016", 2016),
("2017", 2017),
("2018", 2018),
]
name = StringField("Glacier Name")
lower_bound = SelectField(
"First year", validators=[DataRequired()], choices=year_list
)
upper_bound = SelectField(
"Second year", validators=[DataRequired()], choices=year_list
)
class PlotForm(FlaskForm):
name = StringField("Glacier Name", validators=[DataRequired()])
submit = SubmitField("Search")

View File

@ -1,11 +1,10 @@
from app import app, db
from app.forms import LoginForm, YearForm, IntervalForm
from app.models import User, Annual_Data, Glacier
from flask import flash, redirect, render_template, url_for, request
from flask_login import current_user, login_user, logout_user, login_required
from app import app
from app.forms import AnnualForm, LoginForm, PlotForm
from database.queries import query_annual_data, query_plot_data, query_user
from flask import flash, redirect, render_template, request, url_for, send_file
from flask_login import current_user, login_required, login_user, logout_user
from processing.dataframe import create_table, create_plot
from werkzeug.urls import url_parse
from processing.tabulate import create_table
from database.queries import query_annual_data, query_user
@app.route("/")
@ -57,7 +56,7 @@ def data():
@app.route("/table_selection", methods=["GET", "POST"])
def table_selection():
form = YearForm()
form = AnnualForm()
if form.validate_on_submit():
query = query_annual_data(form)
table = create_table(query.statement)
@ -70,6 +69,22 @@ def table():
return render_template("table.html", table=table, title="Table")
@app.route("/plots")
def plots():
return render_template("data.html", title="Data")
@app.route("/plot_selection", methods=["GET", "POST"])
def plot_selection():
form = PlotForm()
if form.validate_on_submit():
query = query_plot_data(form)
plot = create_plot(query.statement)
return render_template("plot.html", title="Plot", plot=plot)
return render_template("plot_selection.html", title="Data", form=form)
@app.route("/plot")
def plot():
return render_template("plot.html", title="Plot", plot=plot)
@app.route("/figure")
def figure(query):
fig = create_plot(query)
return send_file(fig, mimetype="image/png")

BIN
code/app/static/plot.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

View File

@ -4,12 +4,12 @@
<h1 class="text-center">What do you want to check out?</h1>
<li>
<p class="text-center">
<a href="{{ url_for('table_selection') }}">Tables</a>
<a href="{{ url_for('table_selection') }}" class="btn btn-success" role="button" aria-pressed="true">Tables</a>
</p>
</li>
<li>
<p class="text-center">
<a href="{{ url_for('plots') }}">Plots</a>
<a href="{{ url_for('plot_selection') }}" class="btn btn-success" role="button" aria-pressed="true">Plots</a>
</p>
</li>
{% endblock %}

View File

@ -4,5 +4,6 @@
<div class="jumbotron">
<h1 id="igdb-internation-glacier-database">IGDB: Internation Glacier Database</h1>
<p>The IGDB is a database, that uses data from the <a href="http://dx.doi.org/10.5904/wgms-fog-2018-11">WGMS</a> to illustrate the consequences of climate change.</p>
<p>Our system allows you to visualize data with tables and plots, via our intuitive Web UI.</p>
</div>
{% endblock %}

View File

@ -0,0 +1,8 @@
{% extends "base.html" %}
{% import 'bootstrap/wtf.html' as wtf %}
{% block app_content %}
<h1>Plot</h1>
<img src="data:image/png;base64,{{ plot }}" alt="Image Placeholder">
<p><a href="{{ url_for('plot_selection') }}">Back</a></p>
{% endblock %}

View File

@ -0,0 +1,12 @@
{% extends "base.html" %}
{% import 'bootstrap/wtf.html' as wtf %}
{% block app_content %}
<h1>Plot selection</h1>
<div class="row">
<div class="col-md-4">
{{ wtf.quick_form(form) }}
</div>
</div>
<br>
{% endblock %}

View File

@ -23,3 +23,16 @@ def query_annual_data(form) -> BaseQuery:
def query_user(form) -> BaseQuery:
user = User.query.filter_by(username=form.username.data).first()
return user
def query_plot_data(form) -> BaseQuery:
query = (
db.session.query(Annual_Data)
.join(Glacier, Glacier.id == Annual_Data.id)
.filter_by(name=form.name.data)
.group_by(Annual_Data.year)
)
if query.first() is None:
flash("Sorry, no results found")
return redirect(url_for("plot_selection"))
return query

View File

@ -1,6 +1,6 @@
from app import app
from database import db_setup, export, parser
from processing import tabulate
from processing import dataframe
db_setup.main()
parser.main()

View File

@ -0,0 +1,45 @@
from app import db
from io import BytesIO
from pandas import DataFrame, read_sql
from base64 import b64encode
def create_dataframe(query) -> DataFrame:
df = read_sql(sql=query, con=db.engine)
return df
def render_table(df) -> str:
df.fillna(value=0, inplace=True)
table = df.to_html(classes=["table-striped", "table-hover"])
return table
def render_plot(df):
df.fillna(value=0, inplace=True)
plot = df.plot("year", ["surface", "length", "elevation"], kind="bar")
plot_figure = plot.get_figure()
figure = BytesIO()
plot_figure.savefig(figure)
figure.seek(0)
return figure
def encode_plot(plot):
buffer = b"".join(plot)
buf = b64encode(buffer)
encoded_plot = buf.decode("utf-8")
return encoded_plot
def create_table(query) -> str:
df = create_dataframe(query)
html_table = render_table(df)
return html_table
def create_plot(query):
df = create_dataframe(query)
plot = render_plot(df)
encoded_plot = encode_plot(plot)
return encoded_plot

View File

@ -1,19 +0,0 @@
from app import db
from pandas import DataFrame, read_sql
def create_dataframe(query) -> DataFrame:
df = read_sql(sql=query, con=db.engine)
return df
def render_table(df) -> str:
df.fillna(value=0, inplace=True)
table = df.to_html(classes=["table-striped", "table-hover"])
return table
def create_table(query) -> str:
df = create_dataframe(query)
html_table = render_table(df)
return html_table

View File

@ -32,24 +32,23 @@ Requisitos
### Datos
1. **RD1**: Datos del glaciar
- País - *Cadena de 30 caracteres máximo*
- Nombre del glaciar - *Cadena de 30 caracteres máximo*
- ID del glaciar (Compatible con la WGMS) - *Entero de 5 dígitos*
- País - *Cadena de 60 caracteres máximo*
- Nombre del glaciar - *Cadena de 60 caracteres máximo*
- ID del glaciar (Compatible con la WGMS) - *Cadena de 20
caracteres*
2. **RD2**: Datos anuales de un glaciar
- ID del glaciar (Compatible con la WGMS) - *Entero de 5 dígitos*
- Área - *Entero de 10 dígitos*
- Volumen - *Entero de 10 dígitos*
- Grosor - *Entero de 10 dígitos*
- Año - *Entero de 10 dígitos*
3. **RD3**: Datos de cambio de un glaciar
- ID del glaciar (Compatible con la WGMS) - *Entero de 5 dígitos*
- Variación de área - *Entero de 10 dígitos*
- Variación de volumen - *Entero de 10 dígitos*
- Variación de grosor - *Entero de 10 dígitos*
- Año - *Entero de 10 dígitos*
4. **RD4**: Datos del administrador
- ID - *Entero de 4 dígitos*
- Fecha de alta - *Fecha en formato dd-mm-yyyy*
- ID del glaciar (Compatible con la WGMS) - *Cadena de 20
caracteres*
- Área - *Decimal*
- Volumen - *Decimal*
- Altura - *Decimal*
- Año - *Entero de 11 dígitos*
3. **RD3**: Datos del administrador
- ID - *Entero de 11 dígitos*
- Fecha y hora de alta - *Fecha y hora en formato yyyy-mm-dd
hh:mm*
- Nombre de usuario - *Cadena de 20 caracteres máximo*
- Hash de la contraseña - *Cadena de 128 caracteres máximo*
### Funcionales
@ -67,7 +66,7 @@ Requisitos
3. **RF3**: Cálculo de las variaciones anuales
Calcula las variaciones anuales de grosor, área y volumen para un
Calcula las variaciones anuales de altura, área y longitud para un
glaciar
- Entrada: **RD2**
@ -161,9 +160,9 @@ Ingeniería del Software.
![IGDB](./BB.png)
### Diagrama Entidad-Relación
\clearpage
\newpage
### Diagrama Entidad-Relación
![](./ER.png)

Binary file not shown.