Wean hung up after upload big json


#1

Hello.

We’ve installed last release of Wekan v0.72 and trying to import big json from Trello. Wekan has imported it and after, when I try to open imported board it were open, but nodejs hung up and wekan didn’t respond.
Nodejs is 8.9.4.
What do I need to do to solve this problem?
If you need I have profiler log of this hung process.


#2

@swood

Probably some or many of these
a) your import did not finish correctly
b) you don’t have enough RAM
c) you use Chrome webbrowser

Please use Firefox webbrowser, it works best with Wekan.


#3

a) Maybe. But wekan don’t write process and implement import in the background.
b) It’s not. 1668 root 20 0 4478084 427352 25100 R 109.9 5.229 34:43.09 node --expose-gc /build/main.js
c) Do you mean doing import in Firefox or regular work?


#4

c) Yes both. Chrome likes to break web apps just for fun:


#5

I’d checked your theory about Firefox and my json wasn’t imported at all.
Also, what about Chrome. We can’t use only Firefox with Wekan.


#6

If there is problem with import, we would need some way to reproduce problem. For example, steps what to do at Trello, and then importing at Wekan would not work, for example if there is problem in JSON that Trello gives at export.


#7

That Chrome bug is not related to import/export I think.


#8

Anyway there is only this one Chrome bug, and it will be fixed.


#9

Please add new issue to https://github.com/wekan/wekan/issues and attach JSON textfile in .zip archive to that issue, so we can fix import.


#10

Unfortunately, I can’t send in the public space this json because it contains private data. But I can tell you count of keys: approximately 28000 pairs.
And the size of json: about 3.6Mb.
And also I have profiler output of nodejs after I tried last time to import this json. I think the very interesting strings there:

[JavaScript]:
ticks total nonlib name
1341 0.6% 5.2% LoadIC: A load IC from the snapshot
759 0.3% 3.0% LazyCompile: * /build/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:390:18

[Bottom up (heavy) profile]:
Note: percentage shows a share of a particular caller in the total
amount of its parent calls.
Callers occupying less than 1.0% are not shown.

ticks parent name
187232 82.6% /opt/nodejs/bin/node
29983 16.0% LazyCompile: * packages/meteor.js:844:25
29981 100.0% LazyCompile: *processImmediate timers.js:697:26
18203 9.7% LazyCompile: *Query.toBin /build/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/commands.js:89:33
18112 99.5% LazyCompile: *Pool.write /build/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:928:32
18110 100.0% LazyCompile: *nextFunction /build/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/cursor.js:518:28
18109 100.0% LazyCompile: *Cursor.next /build/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cursor.js:257:33
18001 99.4% LazyCompile: *fn /build/programs/server/node_modules/fibers/future.js:82:20
17319 9.3% /opt/nodejs/bin/node
5404 31.2% LazyCompile: *stop /build/programs/server/packages/ddp-server.js:2029:22
2726 50.4% LazyCompile: * /build/programs/server/packages/mongo.js:1283:34
2726 100.0% Builtin: ArrayForEach
2726 100.0% LazyCompile: *stop /build/programs/server/packages/mongo.js:1282:20
2678 49.6% Function: ~ /build/programs/server/packages/mongo.js:1283:34
2678 100.0% Builtin: ArrayForEach
1877 70.1% LazyCompile: *.each..forEach packages/underscore.js:136:43
801 29.9% LazyCompile: *stop /build/programs/server/packages/mongo.js:1282:20
2902 16.8% LazyCompile: *_.isEmpty packages/underscore.js:1033:23
2900 99.9% Function: ~stop /build/programs/server/packages/ddp-server.js:2029:22
2900 100.0% Function: ~ /build/programs/server/packages/mongo.js:1283:34
2900 100.0% Builtin: ArrayForEach
1453 8.4% LazyCompile: *Buffer.write buffer.js:855:34
1453 100.0% LazyCompile: *Query.toBin /build/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/commands.js:89:33
1453 100.0% LazyCompile: *Pool.write /build/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:928:32
1453 100.0% LazyCompile: *nextFunction /build/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/cursor.js:518:28
1059 6.1% LazyCompile: *_pollMongo /build/programs/server/packages/mongo.js:2150:24
1059 100.0% Function: ~ /build/programs/server/packages/mongo.js:2116:40
1059 100.0% LazyCompile: * packages/meteor.js:845:20

It’s look like the frontend tries to update of each card in realtime and generating a lot of requests to backend. Does this version make sense?


#11

Are you here? @xet7
Maybe you need any additional information?


#12

Please create new GitHub issue and add your error messages there, because I’m watching GitHub repo and get email notifications about new issues, comments etc immediately. I get notifications from forum with delay or not at all, for example I did not get notification of your debug messages at all.


#13

Yes, sure: