Open trentks opened 7 years ago
manuels
commented
7 years ago Well, I think setting main memory size is fine and the error Either (1) compile with -s TOTAL_MEMORY=X with X higher than the current value 67108864, or (2) set Module.TOTAL_MEMORY before the program runs is the issue.
Did you try to set it lower than 512MB? (Maybe the browser does not allow the size to be that big)
trentks
commented
7 years ago Cheers for the response.
I've tried all of {64MB, 128MB, 256MB, 512MB, 1024MB, 2048MB} all result in the same ! TeX capacity exceeded, sorry [main memory size=250000] message in the logs (working with an unmodified texmf.cnf).
For curiosities sake I also tried {32MB, 16MB, 1MB} and they all result in the same ! TeX capacity exceeded, sorry [main memory size=250000]. So I then tried it with 1kB and then 1B and finally 0B and they all produce the same. So I then tried it without a call to set_TOTAL_MEMORY and I get the same also, see the javascript "compile" function for illustration of this.
I'll include relevant source for completeness below. The basic premise is that there sits a report_totals.tex template file on the server this has a series of place holders for string replacement in php. The php looks up data from a mysql database and should populate these placeholders with tables / graphs. The table is functioning nicely but I want to get graphs too - I have two approaches, the preferred approach is to make these graphs in tex (via php) as this is very simple. The other approach is to make these graphs on HTML5 canvases using a javascript graphing library (I quite like RGraph) then convert these canvases to URI's and put them in the path of the latex compiler (effectively seeing a png image). I'd rather not do the second approach (which I've already implemented for the "histogram.png") as it's a lot more work than doing it in TeX and I have dozens of graphs I'd like to make.
function compile(sourceCode) {
showCompilingIndicator(true);
var pdftex = new PDFTeX("assets/texlivejs/pdftex-worker.js");
pdftex.FS_createLazyFile('/', 'carne.jpg', '../../images/carne.jpg', true, true);
pdftex.FS_createLazyFile('/', 'histogram.png', document.getElementById("month-canvas").toDataURL(), true, true);
pdftex.on_stdout = logCompilation;
pdftex.on_stderr = logCompilation;
pdftex.compile(sourceCode).then(function(pdf) {
showCompilingIndicator(false);
if (pdf) { //pdf === false
window.open(pdf);
}
});
}
function generateReport() {
var species = document.getElementById("species-dropdown").value;
var statistics = document.getElementById("statistics-selection").value;
var date_start = document.getElementById("start-selection").value;
var date_end = document.getElementById("end-selection").value;
var today = new Date();
var date_generated = today.getFullYear() + "-" + today.getMonth()+1 + "-" + today.getDate();
getData("php/report_totals.php", {date_generated: date_generated, date_start : date_start, date_end : date_end, species : species, statistics : statistics}, compile);
}
function getData(url, data, successFunction, completeFunction) {
if (completeFunction === undefined) {
completeFunction = function(){};
}
$.ajax({
type : 'post',
url : url,
dataType : 'json', // expected returned data format.
data : data,
success : successFunction,
complete : completeFunction
});
}This is php/report_totals.php
<?php require_once 'mysql.php';
$date_generated = strval($_POST['date_generated']);
$date_start = strval($_POST['date_start']);
$date_end = strval($_POST['date_end']);
$species = strval($_POST['species']);
$statistics = strval($_POST['statistics']);
$database = "stun_logger_".$species;
$query = "SELECT DATE(dtstamp), total, alarm, ".$statistics."induction, ".$statistics."duration, ".$statistics."and, ".$statistics."nand FROM ".$database.".count WHERE dtstamp BETWEEN '".$date_start."' AND '".$date_end." 23:59:59'";
$rows = $db -> select($query);
$counts_table = "";
foreach($rows as $row) {
$table_row = $row['DATE(dtstamp)']." & ".$row['total']." & ".$row['alarm']." & ".$row[$statistics.'induction']." & ".$row[$statistics.'duration']." & ".$row[$statistics.'and']." & ".$row[$statistics.'nand']." \\\\ \\hline";
$counts_table = $counts_table.$table_row;
}
$report_file = file_get_contents("../tex/report_totals.tex");
$report_file = str_replace("php-date-generated", $date_generated, $report_file);
$report_file = str_replace("php-date-start", $date_start, $report_file);
$report_file = str_replace("php-date-end", $date_end, $report_file);
$report_file = str_replace("php-species", $species, $report_file);
$report_file = str_replace("php-statistics", $statistics, $report_file);
$report_file = str_replace("php-counts-table", $counts_table, $report_file);
print json_encode($report_file);
?>This is tex/report_totals.tex
% replacements to be made in report_totals.php
\newcommand{\dategenerated}{php-date-generated}
\newcommand{\datestart}{php-date-start}
\newcommand{\dateend}{php-date-end}
\newcommand{\species}{php-species}
\newcommand{\statistics}{php-statistics}
\newcommand{\countstable}{php-counts-table}
\documentclass{article}
\usepackage{array}
\usepackage{caption}
\usepackage{fancyhdr}
\usepackage{graphicx}
\usepackage{float}
\usepackage[headheight=30pt]{geometry} %due to picture in header
\usepackage{pgfplots}
\pagestyle{fancy}
\lhead{Stun Totals \datestart{} to \dateend{}}
\rhead{\includegraphics[width=1.2cm] {carne.jpg}}
\begin{document}
\begin{titlepage}
\centering
\includegraphics[width=0.5\linewidth]{carne.jpg} \\[2cm]
\rule{\linewidth}{0.2 mm} \\[0.4 cm]
{\huge \bfseries Stun Totals Report} \\
\rule{\linewidth}{0.2 mm} \\ [2 cm]
\textsc{\huge \textbf{\species{} \statistics{} Analysis}} \\[2cm]
\textsc{\large \datestart{} to \dateend{}} \\[2cm]
{\large Generated: \dategenerated{}} \\[2cm]
\vfill
\end{titlepage}
\begin{center}
\begin{table} [H]
\caption{Total Counts}
\begin{tabular}{| >{\centering\arraybackslash}m{2cm} | >{\centering\arraybackslash}m{1.0cm} | >{\centering\arraybackslash}m{1.0cm} | >{\centering\arraybackslash}m{2cm} | >{\centering\arraybackslash}m{2cm} | >{\centering\arraybackslash}m{2cm} | >{\centering\arraybackslash}m{2cm} |} \hline
\textbf{Date} & \textbf{Total} & \textbf{Alarm} & \textbf{Induction Failure} & \textbf{Duration Failure} & \textbf{Concurrent Failure} & \textbf{Concurrent Success} \\ \hline
\countstable{}
\end{tabular}
\end{table}
\end{center}
\includegraphics[width=0.8\linewidth]{histogram.png}
\begin{center}
\begin{figure} [H]
\centering
\begin{tikzpicture}
\begin{axis}[
title = LF Calculated RMS Contact Voltages,
xticklabel interval boundaries,
x tick label style={rotate=90, anchor=east},
x label style={at={(axis description cs:0.5,-0.85)},anchor=south},
xlabel=Contact Voltage (RMS),
ylabel=Observations,
ybar interval=1,
height = 4.0cm,
width = 7.0cm
]
\addplot
coordinates {(0,0) (0.65,0) (1.29,1) (1.94,7) (2.59,5) (3.23,3) (3.88,1) (4.53,2) (5.17,1) (5.82,0) (6.46,0)}; % place holder (to fill with php)
\legend{}
\end{axis}
\end{tikzpicture}
\label{fig:rms1}
\end{figure}
\end{center}
\end{document}
fzimmermann89
commented
7 years ago did you check if pgflibraryplothandlers.code.tex is corrupted in the virtual file system of emscripten? this is not a problem with memory size, but most likely a tex error
JChoquette
commented
5 years ago @trentks Did you ever find a solution to this issue? I am having an identical issue attempting to use the tikz package.
trentks
commented
5 years ago @JChoquette
I'm afraid to say that I could not resolve this issue.
In the end I had to make my graphs in javascript (Rgraph) on some non-visible canvases, and fed these through in base64 pngs to the report. It really made the reporting look rather ugly with mismatched fonts etc. But it was a compromise that I had to make.
If you do find a solution please let me know, but otherwise I can only suggest that you do similar. Templating code to achieve such a thing is in my code above.
Cheers.
JChoquette
commented
5 years ago @trentks Unfortunately although my problem is the same as yours the end goal is sufficiently different that I don't think your suggested alternative will work. Nevertheless, thank you for your swift reply, this was going to haunt me otherwise.
trentks
commented
5 years ago @JChoquette
No worries mate, please do report back if you have any success as I wasn't entirely happy with my (forced) workaround. It would be very useful to have a solution to this, as there any many other projects for which I'd like to do reporting in LaTeX. Not having Tikz is a real headache for most of them.
Godspeed.
trentks
commented
5 years ago @JChoquette
Did you get anywhere with this?
JChoquette
commented
5 years ago Sorry, no luck :(
Absolutely brilliant idea this, having texlive as a javascript module - magic!
I've managed to configure several packages (
float,titling,caption) into this with great result.Process to add package:
./texlive/texmf-dist/./texlive.lstand./texlive/texmf-dist/ls-Rby running the following (I have this saved as bash .sh file in the./directory)That all seems to work fine. Now the issue is in trying to get pgfplots to run.
Process to configure pgfplots:
pgfplots.tds.zipfrom CTAN./texlive/texmf-dist/./texlive/texmf-dist/ls-Rand./texlive.lst.tikz.stywhich belongs to thepgfpackage)pgf.tds.zipfrom CTANeveryshi.stywhich belongs to themspackage)ms.tds.zipfrom CTANxcolor.stywhich belongs toxcolorpackage)xcolor.tds.zipfrom CTANProblems:
Here's the compile log...
Attempted solutions:
! TeX capacity exceeded, sorry [main memory size=250000].in the log.First attempt is that from the wiki which suggests changing the
pdftex.set_TOTAL_MEMORY(memsize_in_bytes)so here's my javascript call to compile.pdftex.set_TOTAL_MEMORY(512*1024*1024)which is a ridiculously large number and it doesn't change the result./texlive/texmf.cnffile and adding memory parameters so I try something like this (again with large values), this is my./texlive/texmf.cnffilemain_memory,extra_mem_bot,font_mem_size,pool_sizeandbuf_sizevariables.Either (1) compile with -s TOTAL_MEMORY=X with X higher than the current value 67108864, or (2) set Module.TOTAL_MEMORY before the program runs.error message (regardless of mypdftex.set_TOTAL_MEMORYcall)./texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothandlers.code.tex, deleting or modifying this in any way (ie shortening it so that it doesn't occupy so much memory) results in the same error, only after the next file in the sequence that it needs to fetch to compile.texmf.cnfrun some kind of utility calledtexhashto populate sub-dependent files to actually fully configure the memory changes. My issue is callingtexhashalters the local install, not the texlive.js folder I don't know what subchanges are actually needed! Similarly this post suggests similar but withfmtutilneeding to be called - same issue, I don't know how to call this on the texlive.js directory.Conclusions
I don't think anything I've done has impacted the available memory to the compiler, even though I've been able to alter the
[main memory size=250000]to[main memory size=4250000]in the logs (which is 17 times larger) I think this is just a misreporting or I'm not altering the right kind of memory (number of strings, etc). Any ideas?